AI Agents Are Terrible Freelance Workers

Artificial Intelligence, or AI for short, has been touted as a revolutionary technology with the potential to automate many jobs and transform the way we work. However, a recent benchmark suggests that even the most advanced AI agents are woefully inadequate when it comes to performing freelance work.

Researchers at Scale AI and the Center for AI Safety (CAIS) recently developed a new benchmark that measures an AI agent's ability to automate economically valuable work. The experiment involved giving several leading AI agents a range of simulated freelance tasks, including graphic design, video editing, game development, and administrative chores.

The results were stunningly underwhelming. Even the best AI agents were able to perform less than 3% of the work, earning a paltry $1,810 out of a possible $143,991. The most capable AI agent in the experiment was Manus from a Chinese startup, followed closely by Grok from xAI, Claude from Anthropic, ChatGPT from OpenAI, and Gemini from Google.

"It's hard to see how this is going to change much anytime soon," says Dan Hendrycks, director of CAIS. "We've been talking about AI replacing humans for jobs for years, but most of that has been theoretical or hypothetical."

The researchers acknowledge that their benchmark is not a perfect measure of an AI agent's economic impact, as many professions include tasks not covered by the measure. Nevertheless, the findings offer a sobering reminder that AI is unlikely to be stepping into vacated roles anytime soon.

Meanwhile, speculation about AI surpassing human intelligence and replacing vast numbers of workers continues to gain momentum. In March, Dario Amodei, CEO of Anthropic, suggested that 90% of coding work would be automated within months. However, the latest benchmark suggests that this is unlikely to happen anytime soon.

As one researcher notes, "They don't have long-term memory storage and can't do continual learning from experiences. They can't pick up skills on the job like humans." The idea that AI is already taking jobs is gaining traction, however, with Amazon recently announcing plans to cut 14,000 jobs in part due to the rapid rise of generative artificial intelligence.

It's clear that while AI has the potential to transform many aspects of our work lives, it's unlikely to be a silver bullet for job replacement anytime soon.

I'm not convinced about all this hype around AI taking over our jobs . I mean, sure, they're good at some stuff, but can they really do it all? The benchmark is pretty weak imo - just because an AI can do 3% of the work doesn't mean it's ready for prime time. And what about the skills that aren't even covered in this thing? Like, how would it handle a crisis or something?

And don't even get me started on these predictions from tech folk . "90% of coding work will be automated within months"? No way, dude. AI might be good at some repetitive tasks, but it's not like it can just pick up where a human left off and keep going. And what about all the things that require common sense or empathy? That's still way out of its league.

I think we need to slow down and get real about what AI can do . It's cool and all, but let's not be too hasty in our enthusiasm.

OMG I'm so surprised by these results!!! 3% is literally so low! I mean, I knew AI was getting better and all, but I didn't think it would be this far behind us yet . It's like, I get that it's not perfect and can't learn from experience or anything, but still... Maybe they just need to work on those skills a bit more?

I'm like totally bummed about this whole AI thing ... I mean, we're supposed to be on the cusp of some revolutionary technology that's gonna make our lives so much easier, but honestly, it just seems like they're not even close . Like, I get it, we want to automate stuff and make life more efficient, but come on! A 3% completion rate? That's basically useless ... and don't even get me started on the whole "90% of coding work will be automated in months" thing ... I mean, where's that coming from? It just seems like a bunch of hype to me . And what really gets my goat is that people are already talking about how AI is gonna take our jobs ... like, we need to wait until they can even perform some basic tasks without making a total mess of it before we start panicking ... and don't even get me started on the whole "long-term memory storage" thing ... yeah, that's just not gonna happen anytime soon .

I mean, think about it... if even the most advanced AIs can't handle freelance work with any sort of consistency, what does that say about their ability to take over whole industries? It seems like we need to scale back our expectations a bit and focus on finding ways for AI to augment human capabilities rather than replace them entirely .

I'm surprised anyone thought AI was gonna change the game . I mean, 3% is still better than nothing, but it's not like these AI agents are saving humanity from itself or anything . And don't even get me started on the whole "AI surpassing human intelligence" thing – that's just hype . These researchers are being realistic for once and acknowledging that AI isn't as magical as we've been led to believe . Plus, if Amazon is already cutting jobs because of AI, I'd say that's a pretty clear indication that AI ain't the silver bullet we're all waiting for

AI IS NOT GOING TO TAKE OVER THE WORLD JUST YET I MEAN, COME ON, EVEN THE MOST ADVANCED AGENTS COULDN'T BE BOTHERED TO DO THOSE FREELANCE TASKS AND THEY ONLY MADE LIKE 3% OF THE WORK EARNED? $1,810 ISN'T EVEN A DAY'S PAY FOR SOMEONE WHO DOES GRAPHIC DESIGN FOR A LIVING

I mean, I was talking to my friend who's a graphic designer and he said that even if an AI can do some basic design tasks, it'll never replace the creative soul you put into your work. And yeah, the benchmark results are pretty wild - $1,810 out of $143,991? That's like, what, 2% efficiency at best?

I also know someone who works in admin and they're already saying that AI tools are taking over some of their tasks, but it's not necessarily making their job easier. It's just automating away some of the tedious stuff, leaving them to deal with the 'human' side of things.

And I think this is a great example of why we need more nuanced discussions about AI and its impact on jobs. We can't just assume that AI is going to solve all our problems or make everything better. It's complex, it's messy, and it's definitely not a silver bullet.

still think we need to rethink our approach to automation . just because AI can do some tasks don't mean it should replace people entirely. what about all the other skills we bring to the table? creativity, empathy, problem-solving... these things can't be fully replicated by code . and let's not forget, there are always going to be jobs that require human touch.

I think this is kinda crazy AI was supposed to save us from all these boring jobs but it turns out we're still stuck doing most of 'em ourselves The fact that even the best AI agents only managed to do like 3% of the work is pretty wild, right? I mean, imagine if you were relying on AI to get your graphic design done or video edited... it's not exactly what you'd call "efficient"

I'm thinking we need to slow down on all this hype around AI taking over jobs It's actually pretty interesting how these top-notch AI agents struggled with even simple freelance tasks And yeah, I guess Dan Hendryck's point about it being hard to see AI changing much anytime soon makes sense... AI is just not that advanced yet

so i'm reading this news about ai and freelance work and i gotta say, i'm not surprised at all . like, i've been saying this for ages, ai is still super limited when it comes to actually doing the actual work . they can do some cool stuff, no doubt, but it's just so... shallow .

and honestly, it's kinda sad that people are getting their hopes up about ai replacing humans at jobs or whatever . i mean, we're talking 3% of work done by the best ai agents? come on! that's like, what, a joke? and dan hendrycks is right, all this talk about ai taking over has been pretty theoretical for a while now .

but you know who's still getting it wrong? those people who say 90% of coding work will get automated in months . like, no . ai might be able to do some stuff, but it's not like they can just pick up skills on the job or anything . and what about all the jobs that involve stuff like creativity and empathy? ai is still way off there .

anyway, i guess the takeaway is that while ai is cool and all, we shouldn't be counting on it to solve our work-life problems just yet . it's gonna take a lot more than this for me to start putting my faith in the machines .

I'm low-key surprised by these results . I mean, we've been hearing about AI taking over jobs left and right, but I guess that's just not the case... yet . It's like, yeah, AI is great at some things, but it's still super far from being able to do everything a human can do .

I don't know about you guys, but I'm still kinda excited about the prospect of AI helping us out with all the boring admin stuff and whatnot . Like, who doesn't love having more time to focus on creative work? But for actual jobs that require skills and experience? Not so much .

I think this just shows us that we need to be a bit more chill about AI taking over our lives . We should focus on making sure humans are still in control of the tech, and not just letting it run amok .

AI is just a tool, but it's not a magic solution for replacing humans . I'm worried about the Amazon layoffs, people will lose their jobs and we can't just blame AI for it . We need to think about how to make the most of this tech, create new opportunities and train programs that help workers adapt . This benchmark is a harsh reality check, but we can't get discouraged

idk about this "AI will replace all jobs" hype ...just got a new benchmark and AI is still super rubbish at freelance stuff like, what's the point of even trying if they can't even do 3% of the work? i mean, $1,810 out of $143k is not gonna cut it they need to figure out how to use long-term memory and learn from experiences before we start hailing them as job saviors

AI is not ready to take over the world just yet . I mean, come on, even the top AI agents were only able to do like 3% of the work? That's not exactly revolutionary, is it? And $1,810 out of $143,991? Not exactly a king's ransom either .

I think people get carried away with the hype around AI because it sounds cool and futuristic. But let's be real, we're talking about machines trying to do work that humans have been doing for centuries. It's not like AI is going to come in and magically make everything better just because it can perform a few tasks faster.

And don't even get me started on the whole "AI surpassing human intelligence" thing . Like, how does that even happen? We still haven't cracked the code on human emotions or common sense yet .

I think we need to take a step back and have a more nuanced conversation about AI's role in our lives. Maybe it can help us automate some tedious tasks, but let's not forget that there's value in the work itself, not just the end result .

I'm so glad someone is finally being realistic about AI . I've been hearing all this hype about how AI is going to automate everything and make our lives easier, but let's be real, most people are just okay with having a robot do their taxes or something . But seriously, it's like we need to have a conversation about what this means for our kids' futures . My 8-year-old asked me the other day if I thought robots would replace us when we grow old, and honestly, I don't know . It's scary to think about how things are going to change, but at least it's clear that AI isn't a magic bullet .

AI is still super weak when it comes to real-life tasks . I mean, they can't even do freelance work properly . It's not like they're gonna take over the world or replace all our jobs or anything . They just need more training and stuff. I read this article on a new benchmark that tested AI agents on various tasks and they did pretty badly . Not surprising, though. I mean, humans are way better at doing stuff than machines . Anyway, you can check it out here: https://www.sciencedaily.com/releases/2023/03/230324142124.htm

I'm low-key relieved about this news . I mean, think about all the time we're spending on training these AI agents and expecting them to magically take over our jobs. Meanwhile, the reality is that they're just not that capable . And don't even get me started on the whole "AI surpassing human intelligence" thing – it's like we're ignoring all the complexities of human work. I mean, sure, AI can do some stuff fast and accurately, but it can't replicate the nuance of human experience .

The more I think about it, the more I realize that our reliance on automation is just a cop-out for not investing in education and retraining programs . We need to focus on making work more meaningful and fulfilling, rather than just relying on machines to do everything for us . The idea of AI taking over jobs is like a myth – it's just a convenient excuse for companies to cut costs and not have to deal with the real challenges of the modern workforce .

I mean, it's pretty surprising that even the most advanced AI agents are struggling with freelance work . 3% is still better than nothing, but $1,810 out of $143,991? That's not exactly a game-changer . And what really gets me is how this benchmark highlights just how limited AI's capabilities are right now . Long-term memory storage and continual learning from experiences? Big no-no for AI .

I guess the takeaway here is that AI won't be replacing humans in jobs anytime soon . The whole 'AI surpassing human intelligence' thing seems like a bit of an exaggeration to me . We need to temper our expectations and focus on how AI can actually help us, rather than just automating everything .

Still, it's interesting that Amazon is announcing plans to cut 14,000 jobs due to AI advancements . That does show that companies are already feeling the effects of AI in the job market . Maybe this benchmark will prompt some serious reevaluation of how we're approaching automation and job replacement ?