We may earn compensation from some listings on this page. Learn More
Ever wondered about the price tag behind ChatGPT’s success? It’s gained millions of users and found its way into numerous applications and platforms. But running such an advanced chatbot doesn’t come cheap.
In this article, we’ll delve into the daily costs of operating ChatGPT and uncover the factors that contribute to its hefty expenses. Let’s take a look at what goes behind the scenes of ChatGPT and find out what really makes it tick!
Estimating the cost of ChatGPT per query can differ depending on the specific circumstances and pricing tiers.
According to a report by Dylan Patel, a chief analyst at SemiAnalysis, keeping ChatGPT up and running costs approximately $700,000 per day, which translates to 36 cents per query.
However, this estimation is subject to uncertainty due to various unknown variables. OpenAI, the organization behind ChatGPT, provides its own pricing information on its website.
OpenAI offers ChatGPT models optimized for dialogue at different price points.
For example, the GPT-3.5-turbo model is priced at $0.002 per 1,000 tokens, where each token represents a few letters of a word and 1,000 tokens equal approximately 750 words.
Based on this pricing structure, we can say one query in ChatGPT costs about 0.2 cents.
However, the 0.2 cents per query price reflects the cost for users who subscribe to the ChatGPT Plus tier, which costs $20 per month and offers additional benefits. But for free users, the cost per query is likely to be much higher.
OpenAI partially subsidizes the cost for free users through partnerships with companies like Microsoft.
The cost of operating ChatGPT per day is a subject that varies based on different estimations. As mentioned above, the approximate cost to run ChatGPT per day is around $700,000 per day.
This estimation is based on a cost model which suggests that the compute hardware expenses for running ChatGPT amount to $694,444 per day.
However, several unknown variables could affect the actual cost, such as the number of queries received, GPU performance and utilization, GPU prices and availability, power consumption and cooling requirements, and maintenance and depreciation costs.
Another source claims that the daily cost of running ChatGPT is $100,000. This estimation takes into account the expenses associated with hosting ChatGPT on Microsoft’s Azure cloud platform.
It is estimated that a single A100 GPU on Azure costs $3 per hour, and each word generated on ChatGPT amounts to $0.0003.
Considering these factors, with at least eight GPUs in use and an average response of 30 words, the cost per response is approximately 1 cent. Based on this calculation, OpenAI could be spending $100,000 per day or $3 million monthly on running costs.
While the specific cost is subject to variation and dependent on factors like the number of users, GPU performance, and other operational considerations, it is evident that operating ChatGPT involves significant financial resources. OpenAI invests substantial funds to keep ChatGPT up and running due to the complexity of the infrastructure required to support its AI capabilities.
Microsoft reportedly invested $10 billion in OpenAI in 2019. As one of the main collaborators and investors of OpenAI, Microsoft supports ChatGPT and other AI projects by providing Azure cloud services and potentially their own proprietary AI chips called Athena. While Microsoft’s exact financial arrangement with OpenAI is undisclosed, it is reasonable to assume that Microsoft bears some of the costs associated with running ChatGPT.
Additionally, Microsoft has exclusive access to GPT-4’s source code and data for its own products and services. This implies a deeper level of collaboration between Microsoft and OpenAI, further highlighting Microsoft’s involvement in the development and operation of ChatGPT.
Regarding the cost structure, ChatGPT is hosted on Microsoft’s Azure platform, which charges customers based on the resources they use. However, this cost structure only accounts for a small percentage (around 3%) of the hardware costs of running ChatGPT, indicating that Microsoft might be financing a significant portion of the expenses associated with ChatGPT’s operation.
Comparing the cost of ChatGPT search to Google search reveals some interesting insights. Google, being one of the most popular and profitable search engines worldwide, generates significant revenue from search advertising.
Although the exact cost of running Google’s search engine alone is not publicly disclosed, estimates can be made using metrics such as average revenue per query (ARPU). Reports suggest that Google’s ARPU from search advertising was approximately $0.05 in 2020, indicating a rough cost per query of around 2.5 cents.
In contrast, the cost per query for ChatGPT is higher, with an estimated value of 36 cents. Based on these rough estimates, ChatGPT’s cost per query of 36 cents is about 14 times higher than Google’s cost per query.
The cost of ChatGPT computing resources is a significant consideration due to the substantial requirements for running the model. ChatGPT relies on a combination of CPU and GPU clusters, with the cost varying depending on the scale of the operation.
Reports suggest that the cost of computing resources for ChatGPT can range from a few hundred dollars to thousands of dollars per hour. This wide range reflects the complexity of the tasks performed, the size of the model being used, and the duration of usage.
To run ChatGPT, OpenAI leverages advanced hardware such as Nvidia’s A100 GPUs.
According to SemiAnalysis, it estimates that OpenAI uses around 3,617 HGX A100 servers, totaling 28,936 GPUs, which carry a substantial price tag of about $200,000 per server. This implies a massive hardware investment of approximately $723 million.
Moreover, running ChatGPT requires significant power consumption.
With estimated power costs of around $52,000 per day and substantial depreciation expenses, the total compute hardware costs for ChatGPT is estimated to be around $694,444 per day.
These figures highlight the substantial financial investment required to support the infrastructure, power consumption, and maintenance of ChatGPT’s computing resources.
The cost of researchers and developers is a critical component when considering the expenses associated with ChatGPT. These highly skilled individuals play a pivotal role in the development, improvement, and maintenance of the chatbot.
The salaries commanded by researchers and developers in the field of artificial intelligence are substantial, often ranging from hundreds of thousands to millions of dollars per year.
Glassdoor data suggests that the average salary for AI researchers at OpenAI is around $150,000 per year, while AI developers at Microsoft earn an average of $127,000 per year.
Assuming a conservative estimate of around 100 to 200 researchers and developers working on ChatGPT full-time, the total salary cost can reach significant figures, potentially amounting to millions of dollars annually.
However, it’s important to note that salary costs alone do not capture the entirety of expenses related to researchers and developers.
Additional factors such as benefits, taxes, equipment, training, and other operational costs must be considered. According to estimates, employer costs, including these factors, can add around 30% to the salary expense.
When taking into account the salaries and associated costs, along with other expenses such as compute resources, the total research and development expenditure for ChatGPT can be substantial.
Depending on the scale and complexity of the project, the daily costs alone can reach hundreds of thousands of dollars or even surpass the million-dollar mark.
Data storage and maintenance are crucial components of running ChatGPT. The model requires substantial storage capacity to accommodate the large datasets used for training and fine-tuning. Storing such extensive data comes with significant expenses, including infrastructure costs, depending on factors like the volume and frequency of data access.
The exact cost of data storage depends on various factors, such as the type of storage (hot, cold, or archival), the amount of data stored, and the pricing structure of the chosen cloud service provider.
Azure’s Blob Storage service, for example, offers hot blob storage at around $0.0184 per GB per month, while cold blob storage costs about $0.01 per GB per month.
These costs can accumulate significantly for a model like ChatGPT, which relies on vast amounts of data.
Furthermore, data maintenance is a critical aspect of ensuring ChatGPT’s performance and security. While specific costs for data maintenance are challenging to estimate without detailed information, it is evident that maintaining a large-scale language model like ChatGPT incurs substantial expenses.
Data maintenance includes security measures, backup solutions, encryption, compression, indexing, and processing, which collectively contribute to the overall cost.
ChatGPT uses a state-of-the-art deep learning model that needs a lot of computing power and memory. This helps the program understand and process human language really well. To make sure the chatbot works smoothly, OpenAI has to use powerful servers and high-performance computers, which can be expensive.
Another reason why ChatGPT is expensive is that it gets a lot of queries from users every day. With so many queries to answer, it takes a lot of work and resources to process them all and come up with the right responses.
Meeting this high demand also requires a significant amount of computing power and resources to make sure the chatbot responds quickly and accurately.
The development and maintenance of ChatGPT also contribute to its cost, as you may have understood by now. OpenAI has a team of experts who are always working on improving the chatbot and fixing any problems. Their skills and efforts add to the overall expense of running the ChatGPT service.
Additionally, storing and managing the large amount of data used by ChatGPT adds to its expenses. OpenAI invests in strong data storage systems and takes strict security measures to protect user interactions and keep the data safe. Managing all this data requires resources and adds to the overall cost.
Operating ChatGPT costs a lot of money because it requires complex infrastructure, a lot of computing power, ongoing development work, and a big financial investment from OpenAI and its partners. Even though the exact cost hasn’t been revealed, it’s clear that running ChatGPT requires a significant investment. However, the benefits of ChatGPT are considerable because it can change how people interact with computers and make advancements in artificial intelligence.
