I came across this tool via Sasha Luccioni on LinkedIn. I started following Dr. Luccioni’s work after listening to Mystery AI Hype Theater 3000, Episode 19 – The Murky Climate and Environmental Impact of Large Language Models. That quote about “how much water getting ChatGPT to write your email” you’ve probably seen all over social media comes from one of their papers. It’s an interesting discussion and is worth a listen.
Anyway, this tool is really interesting. As the response to your prompt appears it tallies total energy use, an estimate compared to different appliances you might be more familiar with (e.g. charging a phone – click on it to see another comparison), and the duration the response ran for. Note that this tool is currently running for Mistral and Qwen large language models, and that every model will use different amounts of energy. To really know how much energy your ChatGPT, Gemini, or Claude conversations use, we would need transparency from the companies managing those models. HuggingFace is really leading the way here.
Photo by Michael Pointner on Unsplash
2 responses to “Ever wonder how much energy your chats with LLMs use?”
That’s rather slick- but do you end up burning the energy to get the estimate? I also wonder at the simplicity of such approaches- per query. I don’t LLM much but when people tout their successes I suspect they do not reveal how many reprompts it takes to get there or how much time spent in the process. I failed to get any answers searching for an average number of prompts in a session (nearly all results are prompt engineering tips).
I was in an online discussion on Ai a few weeks back when energy use came up as an issue, and Lance Eaton raised a point that stayed in my mind. It does not dismiss the environment impact of the Prompt Game but he asked how often we (like people in that webinar) give a thought to the impact of a zillion Zoom person hours a day (plus Teams, FaceTime)? Or going farther do we take stock of the impact of streaming movies?
I’ve been in so many Ai sessions where a speaker does the energy use acknowledgement but then just blasts on. Acknowledgment with no action does not do much.
Hey Alan,
You do burn energy getting the estimate, which made me wish I’d chosen a different example when I saw how long the response was (whoops). If I’m reading the wondering correctly, I think this tool could get at that at an individual user basis. The social media posts vary from “every prompt you give ChatGPT a bottle of water evaporates” to “the back and forth creating an email” to “10-20 prompts” etc. As far as I can see the tool keeps a running tally per conversation.
Lance’s point is one I’ve had myself – was in one conversation where someone was raging about the waste of energy on LLMs only later admitting to streaming music via youtube and in 4K (we were talking about data plans); no self reflection in sight about that energy use – and Dr. Luccioni talks about this exact thing elsewhere. It’s one of the reasons I really appreciate her views, she gets into nuances others don’t. It’s her paper that looked at the energy use of one specific LLM she was involved in making that got quoted in the paper that lead to the social media campaign of the water use (and she expresses some displeasure with how her work was misrepresented in said paper).
In one of the interviews I saw with her it was about making informed decisions and about transparency. We have pretty good data on the energy use of those other forms of media Lance mentioned, but poor information from Google and co about their LLMs (and data centers more broadly – see the Data Vampires series from Tech Won’t Save Us with Paris Marx). One of the examples Sasha gives is using LLMs as calculators, which sounds ridiculous for a few reasons, but I see a lot of people now replacing regular search and even calculator use with these tools. Choosing the right tool for the job is one of those underlying messages I think.
The energy use counter makes me think a bit of the warning labels on tobacco products. The research there shows some efficacy to the approach, although the efficacy of the really gross looking images is the highest. I don’t think we will have a comparable approach for LLMs chats, but it’s interesting none the less.
Acknowledgement with no action does not do much – couldn’t put it better myself.