Some neural networks are very expensive to train. This led to the popularization of an approach known as pre-training, whereby a neural network is first trained on a large general-purpose dataset using significant amounts of computational resources, and then fine-tuned for the task at hand using a much smaller amount of data and compute resources.

Expensive to train?

The use of pre-trained networks allows a startup, for example, to build a product with much less data and compute resources than would otherwise be needed if starting from scratch. This approach is also becoming popular in academia, where researchers can quickly fine-tune a pre-trained network for a new task, and then publish the results.

Pre-trained networks give smaller teams a leg up


Pre-training has continued to evolve with the emergence of foundation models such as <a href="https://ai.googleblog.com/2018/11/open-sourcing-bert-state-of-art-pre.html">BERT</a>, GPT, <a href="https://openai.com/blog/dall-e/">DALL-E</a>, <a href="https://openai.com/blog/clip/">CLIP</a>, and others. These models are pre-trained on large general-purpose datasets (often in the order of billions of training examples) and are being released as open source by well-funded AI labs such as the ones at Google, Microsoft, and OpenAI.  They allow startups, researchers, and others to quickly get up to speed on the latest machine learning approaches without having to spend the time and resources needed to train these models from scratch.

Foundational models

One of the risks associated with foundation models is their ever-increasing scale. Neural networks such as <a href="https://github.com/google-research/text-to-text-transfer-transformer">Google’s T5-11b</a> (open sourced in 2019) already require a cluster of expensive GPUs simply to load and make predictions. Fine-tuning these systems requires even more resources. More recent models created in 2021-2022 by Google/Microsoft/OpenAI are often so large that these companies are not releasing them as open source – they now require tens of millions of dollars to create and are increasingly viewed as significant IP investments even for these large companies.

The risks of foundation models: Size & Cost

Dataset alignment can also be a challenge for those using foundation models. Pre-training on a large general-purpose dataset is no guarantee that the network will be able to perform a new task on proprietary data. The network may be so lacking in context or biased based on its pre-training, that even fine-tuning may not readily resolve the issue.  Any startup leveraging foundation models in its machine learning efforts should pay close attention to these types of issues.

The risks of foundation models: Outsourced innovation

The opportunities and risk around using hosted and pre-trained models has led many companies to leverage cloud APIs in the “experimentation phase” to kickstart product development. ﻿Once a company has determined it has a product-market fit, it often transitions to self-hosted or self-trained models in order to gain more control over data, process, and intellectual property. This transition can be difficult, as the company needs to be able to scale its infrastructure to match the demands of the model, as well as manage the costs associated with data collection, annotation, and storage.

The tradeoff 

Foundation models and cloud APIs bring opportunities, risks, and limitations for startups that want to innovate in AI and machine learning.

technologyandthefuture

Technology & The Future

entrepreneurship

Entrepreneurship

startup

Startups

artificialintelligence

Artificial Intelligence

How to Use Massive AI Models (Like GPT-3) in Your Startup

As software and algorithms become an increasingly pervasive part of our lives, there’s growing interest and concern about how they are affecting society, the economy, and politics.Yet, most social studies of algorithms perceive them as obscure black boxes that function autonomously. This isolated look at algorithms, which separates them from their human elements leads us to the wrong understanding and conclusions.

The World Of Algorithms

When a group of computer scientists, researchers, or engineers gather to create an algorithm, they are initially driven by a set of elements.These elements define the problem they want to solve and determine the type of data they need to validate their algorithm.Once the problem is formulated, the researchers must establish the ground truth by collecting the right material that enables them to verify their algorithms and the models they will build in the future.

Elements Of An Algorithm: Ground Truthing

Eventually, an algorithm moves to the programming phase, where a set of modules and a list of instructions are created to solve the defined problem.While this phase is usually reduced to pure source code, there is a lot more to programming than that. The most overlooked aspect of programming practices is all the small provisionary inscriptions that accompany the writing of programs.A better understanding of the whys and wherefores of programming practices would perhaps make it possible to try to infuse this specialness into the algorithmic design communities.

Elements Of An Algorithm: Programming

<ul><li>When an algorithm is implemented and verified against the ground truth, it becomes formulated into a mathematical object that can be later used in other algorithms. </li><li> Once proven, these algorithms become abstracted, taken as proven claims that need no further investigation.</li><li>Algorithms, once formulated, become the foundation of other algorithms, where they will contribute to ground-truthing, programming, formulation, and other practices. </li><li>They become the basis and components of other algorithms and contribute to further work in science.</li></ul>

The Formulation Of The Algorithm

This was always in our minds but we didn't know whom to ask.

Most social studies of algorithms perceive them as obscure black boxes that function autonomously. This isolated look at algorithms leads us to the wrong understanding and conclusions.

computerscience

Computer Science

softwareengineering

Software Engineering

Here’s how algorithms are made

Google is the most powerful tool in the world if you know how to use it properly. Let me show you how to get better at Googling.

How to use Google - Search tips for better results

You can use quotes to force an exact-match search. This is super useful if you know exactly the search phrase you're looking for. You'll simply get more accurate results if you do this.eg. "what is javascript"

Use Quotes for Exact-Match Search

The AND operator will return only results related to both terms. In the example below you can see the perfect use-case of this operator.eg. html AND css

Use the AND Operator

You can use the OR operator to get results related to one of the search terms you enter.eg. (javascript OR python) free course

Use the OR Operator

The -﻿ operator will exclude results that contain a term or phrase. Let's say we want to get JavaScript-related results, but exclude any CSS results from them.eg. javascript -css

Use the - (minus) Operator

You can use the (*) wildcards as placeholders, which will be replaced by any word or phrase.You can use it when you know almost the whole search phrase. Just replace the part you don't know with an asterisk. eg. "how to start * in 6 months"

Use (*) wildcards as placeholders

If you want to search for something in particular on a specific website, this should be your approach.eg. site:freecodecamp.org

Search Inside a Single Website

You can also use this very useful feature to help you find a specific file type. It's handy if you're looking for some materials like PDFs. If you haven't used it before, from now on you will.eg. filetype:pdf learn css

Find a Specific File Type

This can be anything at all. If you're looking to buy something that falls within a specific price range, or you're searching for results that include a specific range of years, this is for you.eg. ecmascript 2016..2018

Search for a Range of Numbers

Use the before operator to return only results before a given date.You must provide year-month-day dates or only a year. eg. javascript before:2020

Use the before Operator

Use the after operator to return only results after a given date. You must provide year-month-day dates or only a year. Super useful if you want to exclude outdated results.eg. web development after:2020

Use the after Operator

As someone who has to search a lot of materials for my day to day affairs, this simple but effective hacks are a time saver!

Google is the most powerful tool in the world if you know how to use it properly. Let me show you how to get better at Googling. 1. Use Quotes for Exact-Match Search You can use quotes to force an exact-match search. This is super useful if you know exactly

problemsolving

Problem Solving

How to Use Google – Search Tips for Better Results

Recurrences take place due to ascertainable circumstances and chains of causality.An example is the phenomenon of multiple independent discovery in science and technology, . Indeed, recurrences, in the form of reproducible findings obtained through experiment or observation, are essential to the natural and social sciences; and, in the form of chance observations rigorously studied via the comparative method, are essential to the humanities. If history has lessons to impart, they are to be found par excellence in such recurring patterns. The reason why I like history.

The repetition of similar events in history

In the field of observation, chance favors only the prepared mind." Discovery, like learning, is a phenomena that takes place in the human brain. The brain configures an event as a new pattern based on previous assumptions that were present at the onset of the event.

Chance Observation. 

History repeats itself. it's all about patterns. 

Understand yourself and those around you.

psychology

Psychology

philosophy

Philosophy

history

History

Historic recurrences. 

Artificial intelligence is already everywhere. From Amazon product suggestions to Google auto-complete, AI has invaded nearly every aspect of our lives. The trouble is that AI just isn’t very good. Have you ever had a meaningful conversation with Siri or Alexa or Cortana? Of course not. But that doesn’t mean it always will be this way. Though it hasn’t quite lived up to our expectations, AI is definitely improving.

A Good Thing?

In a utopian version of an AI-dominated future, humans are assisted by friendly, all-knowing butlers that cater to our every need. In the dystopian version, robots assert their independence and declare a Terminator-style apocalypse on humanity. But how realistic are these scenarios? Will AI ever actually achieve true general intelligence? Will AI steal all of our jobs? Can AI ever become conscious? Could AI have free will? Nobody knows, but a good place to start thinking about these issues is here.

The Scenarios

“So you get into this uncomfortable position where you might be forced to recognize that some humans are non-persons and some nonhumans are persons.”“Now again, if you bite the bullet and say I’m willing to be a specialist, being a member of the human species is either necessary or sufficient for being a person, you avoid this problem entirely. But if not you at least have to be open to the possibility that artificial intelligence in particular may at one point become person like and have the rights of persons.”Susan SchneiderProfessor of Philosophy, University of Pennsylvania

Humans & Nonhumans

“I think a lot of people dismiss this kind of talk of super intelligence as science fiction…” “…because we’re stuck in this sort of carbon chauvinism idea that intelligence can only exist in biological organisms made of cells and carbon atoms. As a physicist, from my perspective intelligence is just a kind of information processing preformed by elementary particles moving around according to the laws of physics. And there’s absolutely no law in physics that says you can’t do that in ways that are much more intelligent than humans.”Max TegmarkPhysicist

Super Intelligence

“If we create intelligence, […] unless we program it with the goal of subjugating less intelligent beings, there’s no reason to think that it will naturally evolve in that direction.”“Particularly if, like with every gadget that we invent, we build in safeguards. And we know, by the way, that it’s possible to have high intelligence without megalomaniacal or homicidal or genocidal tendencies because we do know that there is a highly advanced form of intelligence that tends not to have that desire and they’re called women.”Steven PinkerProfessor of Psychology, Harvard University

Women

Will true AI turn against us?

Humans landed first time on the Moon in 1969, and now it's 2022Nasa is planning to send humans to Moon again, after almost half a century

Apollo But Modern

<ol><li>Scheduled launch by 2024-25</li><li>Will also land First Woman and Black person on Moon</li><li>Will launch on the new SLS Rocket, with Orion capsule</li><li>Will setup a Moon base, for future Mission astronauts to stay</li><li>Moon bases will allow in future to slowly start Human colonization on Moon</li></ol>

Features Of The Artemis Program

Here's what NASA is planning with its Artemis program

scienceandnature

Science & Nature

What Is NASA's Artemis Program

Synthetic media (also known as AI-generated media,<a href="https://en.wikipedia.org/wiki/Synthetic_media#cite_note-2"> </a> generative media ,<a href="https://en.wikipedia.org/wiki/Synthetic_media#cite_note-3"> </a> personalized media , and as <a href="https://en.wikipedia.org/wiki/Deepfakes">deepfakes </a><a href="https://en.wikipedia.org/wiki/Synthetic_media#cite_note-5"> </a> ) is a term for the artificial production, manipulation, and modification of data and media by <a href="https://en.wikipedia.org/wiki/Automation">automated </a>means, especially the use of <a href="https://en.wikipedia.org/wiki/Artificial_intelligence">artificial intelligence </a><a href="https://en.wikipedia.org/wiki/Algorithm">algorithms </a>, such as for the purpose of misleading people or changing an original meaning. Synthetic media as a field has grown rapidly since the creation of <a href="https://en.wikipedia.org/wiki/Generative_adversarial_networks">generative adversarial networks </a>, primarily through the rise of <a href="https://en.wikipedia.org/wiki/Deepfake">deepfakes </a>as well as music synthesis, text generation, human image synthesis, speech synthesis, and more.

AI Generated 

Though experts use the term "synthetic media," individual methods such as deepfakes and text synthesis are sometimes not referred to as such by the media but instead by their respective terminology (and often use "deepfakes" as a euphemism, e.g. "deepfakes for text" for natural-language generation; "deepfakes for voices" for neural voice cloning , etc.)Significant attention arose towards the field of synthetic media starting in 2017 when Motherboard reported on the emergence of pornographic videos altered with the use of AI algorithms to insert the faces of famous actresses.

Deepfakes

Fears of synthetic media include the potential to supercharge fake news, the spread of misinformation, distrust of reality, mass automation of creative and journalistic jobs, and potentially a complete retreat into AI-generated fantasy worlds. Synthetic media is an applied form of artificial imagination.

Misinformation 

<ol><li>The robotic creation of music, whether through machines playing instruments or sorting of virtual instrument notes.</li><li>Directly generating <a href="https://en.wikipedia.org/wiki/Waveform">waveforms </a>that perfectly recreate instrumentation and human voice without the need for instruments, MIDI, or organizing premade notes.</li></ol>In 2016, <a href="https://en.wikipedia.org/wiki/Google_DeepMind">Google DeepMind </a>unveiled WaveNet, a deep generative model of raw audio waveforms that could learn to understand which waveforms best resembled human speech as well as musical instrumentation. Other networks capable of generating music through waveform manipulation include TacoTron (by Google) and DeepVoice (by Baidu).

Deepmind

In 2019, voice cloning technology was used to successfully impersonate a chief executive's voice and demand a fraudulent transfer of €220,000. he case raised concerns about the lack of encryption methods over telephones as well as the unconditional trust often given to voice and to media in general.

Fraudulent Vocals

The capacity to generate music through autonomous, non-programmable means has long been sought after since the days of Antiquity, and with developments in artificial intelligence, two particular domains have arisen:

Music Geberation

Natural-language generation bots mixed with image synthesis networks may theoretically be used to clog search results, filling <a href="https://en.wikipedia.org/wiki/Search_engines">search engines </a>with trillions of otherwise useless but legitimate-seeming blogs, websites, and marketing spam.

Nlp Seo Bots

A combination of speech synthesis and deepfakes has been used to automatically redub an actor's speech into multiple languages without the need for reshoots or language classes.An increase in cyberattacks has also been feared due to methods of <a href="https://en.wikipedia.org/wiki/Phishing">phishing </a>, <a href="https://en.wikipedia.org/wiki/Catfishing">catfishing </a>, and <a href="https://en.wikipedia.org/wiki/Social_hacking">social hacking </a>being automated by new technological methods.

Social Hack

Ai currently has no regulation, so it's media has gone unnoticed for a long time. This is a cultural lag I have witnessed through most of these photos. 

personaldevelopment

Personal Development

movies

Movies & Shows

Synthetic media - Wikipedia

Human image synthesis is technology that can be applied to make believable and even <a href="https://en.wikipedia.org/wiki/Photorealism">photorealistic </a>renditions of human-likenesses, moving or still. It has effectively existed since the early 2000s. Many films using <a href="https://en.wikipedia.org/wiki/Computer_generated_imagery">computer generated imagery </a>have featured synthetic images of human-like characters <a href="https://en.wikipedia.org/wiki/Digital_compositing">digitally composited </a>onto the real or other simulated film material. Towards the end of the 2010s <a href="https://en.wikipedia.org/wiki/Deep_learning">deep learning </a><a href="https://en.wikipedia.org/wiki/Artificial_intelligence">artificial intelligence </a>has been applied to <a href="https://en.wikipedia.org/wiki/Media_synthesis">synthesize images and video </a>that look like humans, without need for human assistance, once the training phase has been completed.

Human Image Synthesis

<ul><li class="ql-align-justify">In February 2019 <a href="https://en.wikipedia.org/wiki/Nvidia">Nvidia </a><a href="https://en.wikipedia.org/wiki/Open_source">open sources </a><a href="https://en.wikipedia.org/wiki/StyleGAN">StyleGAN </a>, a novel <a href="https://en.wikipedia.org/wiki/Generative_adversarial_network">generative adversarial network </a>. Right after this Phillip Wang made the website ThisPersonDoesNotExist.com with StyleGAN to demonstrate that unlimited amounts of often photo-realistic looking facial portraits of no-one can be made automatically using a GAN. Nividia's StyleGAN was presented in a not yet <a href="https://en.wikipedia.org/wiki/Peer_review">peer reviewed </a>paper in late 2018.</li></ul>

Does Not Exists?

The whole process of making digital look-alikes i.e. characters so lifelike and realistic that they can be passed off as pictures of humans is a very complex task as it requires photorealistically <a href="https://en.wikipedia.org/wiki/3D_modeling">modeling </a>, animating, <a href="https://en.wikipedia.org/w/index.php?title=Cross-mapping&action=edit&redlink=1">cross-mapping </a>, and <a href="https://en.wikipedia.org/wiki/3D_rendering">rendering </a>the <a href="https://en.wikipedia.org/wiki/Soft_body_dynamics">soft body dynamics </a>of the human appearance.

Digital Look Alikes

Synthesis with an actor and suitable <a href="https://en.wikipedia.org/wiki/Algorithm">algorithms </a>is applied using powerful computers. The actor's part in the synthesis is to take care of mimicking human <a href="https://en.wikipedia.org/wiki/Facial_expression">expressions </a>in still picture synthesizing and also human movement in motion picture synthesizing. Algorithms are needed to simulate laws of <a href="https://en.wikipedia.org/wiki/Physics">physics </a>and <a href="https://en.wikipedia.org/wiki/Physiology">physiology </a>and to map the models and their appearance, movements and interaction accordingly.

Synthesized Actors

Often both <a href="https://en.wikipedia.org/wiki/Physics">physics </a>/<a href="https://en.wikipedia.org/wiki/Physiology">physiology </a>based (i.e. <a href="https://en.wikipedia.org/wiki/Skeletal_animation">skeletal animation </a>) and <a href="https://en.wikipedia.org/wiki/Image-based_modeling_and_rendering">image-based modeling and rendering </a>are employed in the synthesis part. Hybrid models employing both approaches have shown best results in realism and ease-of-use. <a href="https://en.wikipedia.org/wiki/Morph_target_animation">Morph target animation </a>reduces the workload by giving higher level control, where different facial expressions are defined as deformations of the model, which facial allows expressions to be tuned intuitively. Morph target animation can then morph the model between different defined facial expressions or body poses without much need for human intervention.

Modeling And Renderibg

In the late 2010s, <a href="https://en.wikipedia.org/wiki/Machine_learning">machine learning </a>, and more precisely <a href="https://en.wikipedia.org/wiki/Generative_adversarial_networks">generative adversarial networks </a>(GAN), were used by <a href="https://en.wikipedia.org/wiki/NVIDIA">NVIDIA </a>to produce random yet photorealistic human-like portraits. The system, named <a href="https://en.wikipedia.org/wiki/StyleGAN">StyleGAN </a>, was trained on a database of 70,000 images from the images depository website <a href="https://en.wikipedia.org/wiki/Flickr">Flickr </a>. The source code was made public on <a href="https://en.wikipedia.org/wiki/GitHub">GitHub </a>in 2019. Outputs of the generator network from random input were made publicly available on a number of websites.

Machine Learning

Similarly, since 2018, <a href="https://en.wikipedia.org/wiki/Deepfake">deepfake </a>technology has allowed GANs to swap faces between actors; combined with the ability to fake voices, GANs can thus generate fake videos that seem convincing.

Deepfake

This is what early 2010 to 2018 tech was doing under your nose. Now it's much more complicated.  Stay woke. 

product

Product & Design

Human image synthesis - Wikipedia

Everyday, us humans are focused on making a million little choices and over the past twenty years the amount of choices that are related to digital solutions has exponentially increasedOur work, personal, family and social lives have dramatically changed as a result of several waves of technological innovation.

The Future: Human Or AI

Web3 and the Metaverse are ushering in a new era of decentralization of data and community ownership.There are many questions regarding Web2 and where social media has left us and if there are lessons to be learned from Web3.The biggest one is whether Web3 will alter how we design solutions for human experience.A decentralized internet is exciting, but there are still many questions that need to be answered.If we can approach what we've done with <a href="https://www.entrepreneur.com/topic/technology">technology</a> at scale and take it to a whole new level with depth, then we can actually get closer to understanding what people feel.

Web 3.0 And The Human Experience

With excitement buzzing about web3, how can the tech community start to explore how this and emerging technology will impact the human experience?

The Future is Caring About the Human Experience

Having to switch your devices off on airplanes is frustrating, especially when you’re on a business trip and need to stay in touch with the office or flying solo with kids on a long-haul journey. The good news is that inflight WiFi is becoming available on more and more airlines in place of traditional inflight entertainment. But how in the world does it work 35,000 feet up in the air? Well, this is just another piece of <a href="https://www.rd.com/list/airplane-trivia/">airplane trivia</a> you never knew.

Phones On Airplanes: No Longer Switched Off

There are <a href="http://www.connected-altitude.com/services-technology/how-inflight-wifi-works/">two operating systems</a> for airplane WiFi: ground-based and satellite. Air-to-ground WiFi works in a similar way to your cell phone. Airplanes have an antenna located underneath their body, which links up with cell towers. As the aircraft travels, it simply connects to the nearest transmitter on a rolling basis. The airplane becomes a hotspot, so passengers can do everything they would normally do when connected to the Internet, including sending emails, making calls, and even streaming movies.

Two Different Operating Systems

However, this system can’t work when the plane is flying over large expanses of water, like on transatlantic routes. That’s where satellite comes in.Satellite WiFi uses a network of orbiting satellites to allow a connection. The satellite is linked to ground stations, and the airplane connects using a satellite antenna on the top of the fuselage. Again, the plane uses whichever satellite is nearest as it travels. Satellite WiFi operates on two different bandwidths: narrowband and broadband. Both allow passengers full Internet access though broadband is better.

Doesn't Work On Water

The number of <a href="http://www.letsflycheaper.com/blog/airlines-2/airlines-that-offer-inflight-wifi/">airlines offering inflight WiFi</a> is increasing rapidly. You may need to book directly with the airline, and some operators require a <a href="https://www.gogoair.com/list-all-products">single-use pass or package</a> from a provider such as Gogo. This is also a useful option if you fly frequently with different airlines. The price is usually higher for greater bandwidth, but for your company, it may be worth the investment. Keep an eye out for these <a href="https://www.rd.com/list/airplane-features/">hidden airplane features</a> you had no idea existed either.

Book WiFi Directly

Inflight WiFi connectivity is poised to improve across the board. Technology such as Gogo’s 2Ku system is steadily improving reliability, which is enabling more airlines to introduce WiFi for both domestic and international flights. <a href="https://www.jetblue.com/flying-with-us/wifi">JetBlue</a> offers standard <a href="https://www.rd.com/article/how-to-get-free-wifi/">free WiFi</a> service on all domestic flights, and Air Alaska offers <a href="https://www.alaskaair.com/content/travel-info/flight-experience/inflight-entertainment/text-messaging">free texting over WiFi</a> on all flights as well as the promise of <a href="https://www.alaskaair.com/content/travel-info/flight-experience/inflight-entertainment/wifi">expanded, faster WiFi</a> with Gogo’s 2Ku system through 2020. As of now, most Air Alaska flights offer basic WiFi services on flights in North America.

Improvements To Come

Ever wonder how cell signals travel to 30,000 feet so your child can watch 'Frozen' for the hundredth time? Here's how inflight WiFi can keep you connected.

How Does Inflight Wifi Work, Anyway?

Find 100+ Facts about Technology - Curated by others from trusted sources like Books, Podcasts & Articles - Genuine Insights from like-minded Users like You!