GoDaddy has 50 large language models; its CTO explains why

A year ago, GoDaddy didn’t have a single large language model running with its backend systems. Today, the internet domain registry and web hosting firm has more than 50, some of them dedicated to client-side automation products while others are being readied for pilot projects aimed at creating internal efficiencies for employees.

The first of the company’s generative AI initiatives was to build an AI bot that could automate the creation of company design logos, websites, and email and social media campaigns for the small businesses it serves. Then, earlier this year, it launched an AI customer-facing chatbot, GoDaddy Airo. With a culture of experimentation, GoDaddy has moved to formalize the way it documents more than 1,000 AI experiments to help drive innovation. Because “innovation without some kind of hypothesis and some kind of measurement is novelty,” said GoDaddy’s CTO Charles Beadnall.

Beadnall has led the GoDaddy engineering team’s pivot to building AI solutions; he spoke to Computerworld about those efforts, and challenges. The following are excerpts from that interview:

Tell us about your AI journey and how others can learn from your experience. “We’ve been focused on AI for a number of years. We’ve used different variants of it. AI’s a big term and it’s got lots of different subcomponents: machine learning, generative AI, etc. But what we’ve been focused on over the past several years is building out a common data platform across all of our businesses, such that we have inputs coming from our different interfaces and businesses so that we can understand customer behavior better. That’s really enabled us, along with a culture of experimentation, to really leverage generative AI in a way that we can measure the benefits to our customers and to our bottom line, and do that in a way that we continue to iterate against it.

GoDaddy

“We’re all about delivering results, either to our business and our bottom line or to our customers, and so we want to have a measurable hypothesis or what it is that generative AI will deliver to those. That’s something we’ve been building out over the past several years with common data platforms, a culture of experimentation and now leveraging generative AI in practice.”

How important is it to have measurable deliverables with AI deployments? “Ultimately, if you don’t know what it is you’re going to expect to deliver and have some way of measuring it — it may be successful, but you won’t know that it is. It’s been really important for us to have that controlled A/B test, such that we launch a new feature and measure the results against that. So, if you don’t have some form of data you can measure, whether that’s purchase conversion or product activation or something of that nature…, you won’t really know whether they’re having the intended benefit.”

Do you have to create new data lakes or clean up your data repositories before implementing generative AI? I’ve often heard the refrain, garbage in, garbage out. “There is definitely significant implications here. It’s definitely a concern people need to be aware of. The majority of the quality assurance is being performed by the large language model vendors.

“What we’ve done is built a common gateway that talks to all the various large language models on the backend, and currently we support more than 50 different models, whether they’re for images, text or chat, or whatnot. That gateway is really responsible both for implementing the guardrails…, but also to evaluate the responses back from the LLMs to determining if we’re seeing some kind of pattern that we need to be aware of showing it’s not working as intended.

“Obviously, this space is accelerating superfast. A year ago, we had zero LLMs and today we have 50 LLMs. That gives you some indication of just how fast this is moving. Different models will have different attributes and that’s something we’ll have to continue to monitor. But by having that mechanism we can monitor with and control what we send and what we receive, we believe we can better manage that.”

Why do you have 50 LLMs? “This space is moving at a rapid pace with different LLMs leapfrogging each other in cost, accuracy, reliability and security. The large majority of these are in use in sandbox and test environments with only a very small number currently run in production behind Airo. Some of these will be dropped and never make it to production and others will be deprecated as newer models prove more accurate or more cost effective.”

Can you tell me about this gateway. How does it work and did you build it, or did you get it through a vendor? “It’s something we built and it’s going on a year now. We built it to manage the uncertainty of the technology.

“It started out with our initial push into the space as a way to coordinate among the different LLMs. If you think about it logically, a year ago there was one vendor [OpenAI] but it was clear this was going to be a very exciting space. There were going to be a lot of companies that wanted to get into this space, and so we don’t know who’s going to win. And, I think it’s probably a more nuanced discussion of who’s going to win for what. It may be that one model is better for images and another is better for chat. Still another model is better for text. This is going to evolve in such as way that vendors are going to leapfrog each other. So the gateway is a way for us to be somewhat agnostic to the underlying model that we’re using and adapt quickly in changes to cost and changes in accuracy on that path.”

How did you approach training your workforce on AI, and perhaps more importantly, how did you get them to engage with the technology? “I think that’s been surprisingly easy. We had a business unit that came up with our first use case for it, which is helping customers build content for their site and find the right domain name to put on that site. That’s something that a lot of customers get stuck on initially, because it takes a lot of mental cycles to figure out what domain name you’re going to pick, what content you’re going to put on your site — and if you want to start selling product, you have to create descriptions of those items. So, it’s a customer need that we wanted to address.

“Clearly identifying how AI will help us along a path, that business unit really made it a top priority and surged resources against it to come up with some of our first tests within this space. That really did help the team rally behind it to have that clear, compelling use case. We’re running these tests and getting data back and not every experiment was successful. We’re learning things along the way.

“In some ways, experiments that aren’t successful are some of the most interesting ones, because you learn what doesn’t work and that forces you to ask follow-up questions about what will work and to look at things differently. As teams saw the results of these experiments and saw the impact on customers, it’s really engaged them to spend more time with the technology and focus on customer outcomes.”

Is AI ready for creating real-world products you can sell to clients? Or is it more of an assistant, such as suggesting textual content, checking code for errors, or creating video? “We think it’s definitely ready for prime time. Now, it really depends on what the use case is. This is where I think being able to test in a way you can determine [whether it’s] ready for prime time in this particular usage scenario. But it’s definitely adding value to customer interactions, because it’s a set of steps they don’t need to take, but a majority of our customers are leveraging. There are lots of different use cases. Use cases that require deep expertise, it will continue to get better. If the customer wants assistance in completing something more routine…, that’s certainly a prime candidate for leveraging AI.”

What is GoDaddy Airo? What does it do? “It’s basically the AI enablement of our products and services. It’s our underlying AI technology built on top of our data platform, built on top of our experimentation platform and gateway we’re leveraging against our LLMs. Over time, it may turn into additional new products, but right now we’re focused on it making the products we already sell today that much better. It will evolve over time as we experiment our way into it.”

Do your clients use Airo, or do you use it and offer your clients the AI output you receive? “Basically, as soon as you buy a domain name and website, we’ll jump you directly into that experience. We’ll help you build out a site and if you upload inventory items to it, Airo will automatically fill that [textual] description for you. If we can get them from having an idea to having a live business online, that’s our major objective. That’s where we’ll be rewarded by our customers. That’s our focus. We do have a metric we track for improving the customer’s value and achievement. It’s still early innings there, but we are improving our customers’ ability to get their businesses up and running.”

How accurate is Airo? “I think it’s reasonably accurate. We run experiments where we have a threshold of accuracy, which is relatively high. We wouldn’t be promoting something that didn’t have significant accuracy and [was] benefiting our customers. I’d say it’s been surprisingly accurate most of the time. Again, there are permutations where we continue to learn over time, but for the core experience, so far, it’s proven to be more accurate than we would have expected.”

Where did you obtain your LLMs that power the generative AI? “The actual LLMs we’re using…are ChatGPT, Anthropic, Gemini, AWS’s Titan. So, we are leveraging a number of different models on the backend to do the genAI itself. But all the integrations into our flows and products is the work we do.”

What are some of the barriers you’ve encountered to implementing AI within your organization, and how did you address them? “We moved quickly but also thoughtfully in terms of understanding the security and privacy ramifications. That’s the area I’d say we spent a reasonable amount of time thinking through. I think the biggest barriers is having the creativity in deciding where these LLMs can be applied and how do you design the experiments to address those needs? Basically, building out the capabilities. That’s where we spend our time today with a common platform approach, which can then account for the security.

“It’s easy to spend enormous amounts of money without much benefit. So it has to be about those factors as well as the customer’s needs. Balancing those factors has been a major focus of ours.”

What’s next? “The big opportunity for us is leveraging AI in more places across the company — internally as well as to make our employee experiences more effective and efficient. There’s a lot of territory for us to cover. We’re under way in all the different avenues now. We’ve got a lot of activity going on to finalize how we augment these LLMs with our own data for more internal use cases. We’re in the thick of it right now. We’re identifying which pilot projects to launch internally.”