Google: The AI Company

Discussions on a podcast reveal the deep history of AI at Google, driven by the vision of its founders and a strategic concentration of AI talent, laying the foundations for significant breakthroughs in the field.

Google's Founders and Employees Were Interested In AI

Larry Page's Father, an AI Pioneer, Influenced Google's AI Vision

Larry Page, one of Google's co-founders, envisioned Google as fundamentally an artificial intelligence company from the very beginning. His vision was influenced by his father, who was a computer science professor specializing in AI at the University of Michigan. Page's father had conducted research in the field of AI during its first wave in the 1960s and 70s when it was not a popular computer science field. This had a profound impact on Larry's outlook.

Google Hired Top AI Researchers In the 2000s, Concentrating AI Talent

David Rosenthal and Ben Gilbert discuss how Google concentrated AI talent by employing almost every notable person in AI during the 2000s. This includes hiring luminaries such as Ilya Sutskever, Jeff Hinton, and Alex Krizhevsky, not to mention other prominent figures like Dario Amodei, Andrej Karpathy, Andrew Ng, Sebastian Thrun, and Noam Shazir. These hires found fertile ground at Google to advance their groundbreaking work.

Larry Page hired Sebastian Thurn, then the head of Stanford’s AI Lab, to work on machine learning applications. Thurn suggested that Google should bring in AI academics part-time to work on various projects, which the founders agreed to. George Herrick, an engineer with a machine learning PhD from the University of Michigan, advanced a notion that would foreshadow the development of large language models. Jeff Hinton, a machine learning professor from the University of Toronto, was brought in by Thrun to give a tech talk at Google about new work on neural networks.

Google's Investments in AI Laid Foundations For Breakthroughs

Google Engineers Developed Language Models Like "Fill," a Probabilistic Hierarchical Inferential Learner, Powering Google Products

It was mentioned that Google engineers developed groundbreaking language models with applications across Google’s products. These models demonstrate the capability to understand and respond to human language. Jeff Dean is noted to have used the "Fill" language model to implement AdSense in a week, contributing to billions in new revenue and expanded ad inventory. By the mid-2000s, such models were using significant data center resources due to their wide applications, including predicting search queries and improving ad quality scores.

Acquisition of Deepmind in 2014 Boosted AI Expertise and Research Capabilities

The podcast describes a series of events ...

Google's relentless pursuit of innovation has led to the creation and refinement of AI technologies that have become foundational in the field.

Google's Development of Transformer Architecture

Transformers by Google: Power of Attention and Scalability Demonstrated

Google's development of the Transformer architecture represents a paradigm shift in the AI field. Before the advent of the Transformer, Google Translate had relied on recurrent neural networks, which were limited by short context windows and did not scale efficiently. In 2017, the Google Brain team published a paper describing the Transformer model, which leveraged attention mechanisms to achieve better parallelization and longer context retention. This mechanism mimics how human translators work, understanding the entire context before translating. Due to its elegance, energy efficiency, and scalability, the Transformer model marked the beginning of a modern AI era.

Transformers Were Key In Advancing Large Language Models

The Transformer architecture was not only a milestone for Google Translate but also set the stage for large language models such as BERT, and later influenced models developed by other organizations, like OpenAI's ChatGPT. Recognizing its potential, Google Brain planned to extend the Transformer architecture beyond text to other modalities, such as images and audio. The Transformer model continued Google's legacy in language models and significantly reduced Google Translate's error rate by 60% when it replaced LSTMs in 2016.

Google Developed the TPU for Efficient AI Training and Inference

Google's TPUs: Custom Chips For Scaling Machine Learning

Anticipating the need for high-volume matrix multiplications required by neural networks, Google embarked on the development of the Tensor Processing Unit (TPU). Initiated by Google engineer Jonathan Ross and materialized in a formal project, the custom ASIC was designed to support neural network operations efficiently and at scale.

Google's TPU Architecture and Deployment Gave a Hardware Edge Over Competitors

The TPUs provided Google with a significant hardware edge. Initially kept top secret, TPUs were fundamental in powering projects like AlphaGo. Over time, TPUs evolved, with versions significantly increasing in efficiency. Google's decision to create TPUs internali ...

In the face of burgeoning AI technologies from competitors like OpenAI, Google has taken critical steps to advance its AI development and maintain its position in the market.

ChatGPT and AI Models Threaten Google's Core Business

Since the release of OpenAI's ChatGPT, Google has recognized the serious potential of large language models to disrupt traditional search paradigms. The rapid growth of ChatGPT shows its capability to reshape the way users engage with search engines, challenging Google's core business. Google, aware of the progress within its AI sector, including raw chatbot models like Lambda, responded to the tangible threat by significantly shifting its strategy.

After the launch of ChatGPT, which witnessed unprecedented adoption rates, reaching 1 million users within a week and 100 million by the end of January 2023, Google realized that AI was transitioning from a sustaining innovation to a disruptive one. As such, it posed an existential threat. Microsoft's investment of another $10 billion in OpenAI and its subsequent 100 million registered users for ChatGPT further emphasized the immediacy of the threat.

In response, Sundar Pichai, CEO of Google, issued a "code red" in December 2022, urging the company to expedite the development of native AI products. This urgency led to the release of Bard, which rebranded the Lambda model with a new chatbot interface, following the success of ChatGPT with Bing. However, the initial public release of Bard was met with criticism over its performance.

Google Merges Brain and Deepmind Teams Under one Leader

Sundar Pichai initiated a significant reorganization within Google, directing the merger of Google Brain and DeepMind into a unified team. This decision was aimed at consolidating Google's AI efforts and streamlining capabilities across the company. Demis Hassabis of DeepMind was appointed as the CEO of this integrated AI division, suggesting a concerted effort to centralize Google's AI initiatives.

Following the reorganization, a new high-performance AI system initiative called G ...

As artificial intelligence (AI) continues to evolve, Google is uniquely positioned to benefit from its vast infrastructure and to navigate the challenges AI may pose to its traditional business models.

Google's Infrastructure Might Give It an AI Edge

Ben Gilbert and David Rosenthal unpack how Google’s well-established infrastructure could give the company a significant edge in the competitive AI landscape.

Google's Cost Advantage From Amortizing Model Training Across Users and Services

Google's switch from CPUs to GPUs, orchestrated by Alex Krashevsky, Jeff Hinton, and Ilya Sutskever, led to a transformative $130 million purchase of 40,000 GPUs from Nvidia. This investment, approved by Larry Page, shows Google's understanding of deep learning's strategic importance for its future. Their massive infrastructure has integrated a profound number of GPUs for AI training and inference, providing a likely competitive advantage.

Gilbert emphasizes that Google's unique cloud infrastructure, with revenues of $50 billion, allows the company to spread the substantial costs of model training across its diverse user services. By producing their own Tensor Processing Units (TPUs), and managing all infrastructure in-house, Google achieves a more favorable cost structure for deploying AI at scale.

Running an AI data center incurs substantial costs, primarily from chips and depreciation, but Google stands out as a low-cost provider, with access to low markup hardware. Gilbert mentions that the model training costs are amortized across every Google search, allowing Google to process 10 trillion to nearly 1 quadrillion inference tokens within a surprisingly short period. This vast scale distributes the hardware and training expenses over a significant amount of value creation.

Google's Data Ownership and Personalization Enable Tailored AI Experiences

David Rosenthal notes that Google’s ownership of personalized data from products like Gmail, Maps, Docs, Chrome, and Android allows it to create highly personalized AI products. This edge comes from the fact that Google can use this data in ways that other companies cannot, given they lack access to such abundant and individualized information.

AI Transition May Disrupt Google's Search Model

Despite Google’s strategic preparations for AI, there is a recognized potential disruption to its traditional revenue streams, specifically those tied to web search and advertising.

AI Assistants May Cut Web Searches, Threatening Google's Ad Revenue

The rise of AI-driven products, like OpenAI's ChatGPT, suggests a potential disruption to Google’s search-based ad revenue. Gilbert and Rosenthal discuss how AI advancements, particularly AI assistants, might reduce the volume of traditional web searches. High-value queries, such as those related to trip pl ...