The Pentagon is moving toward letting AI weapons autonomously decide to kill humans
The use of drones capable of deciding whether to kill humans is alarming critics.
Can’t figure out how to feed and house everyone, but we have almost perfected killer robots. Cool.
Oh no, we figured it out, but killer robots are profitable while happiness is not.
What's more important, a free workforce or an obedient one?
Especially one that is made to kill everybody else except their own. Let it replace the police. I'm sure the quality controll would be a tad stricter then
Great, so I guess the future of terrorism will be fueled by people learning programming and figuring out how to make emps so they can send the murder robots back to where they came from. At this point one of the biggest security threats to the U.S. and for that matter the entire world is the extremely low I.Q. of every one that is supposed to be protecting this world. But I think they do this all on purpose, I mean the day the Pentagon created ISIS was probably their proudest day.
The real problem (and the thing that will destroy society) is boomer pride. I've said this for a long time, they're in power now and they are terrified to admit that they don't understand technology.
So they'll make the wrong decisions, act confident and the future will pay the tab for their cowardice, driven solely by pride/fear.
Great, so I guess the future of terrorism will be fueled by people learning programming and figuring out how to make emps so they can send the murder robots back to where they came from.
Eh, they could've done that without AI for like two decades now. I suppose the drones would crashland in a rather destructive way due to the EMP, which might also fry some of the electronics rendering the drone useless without access to replacement components.
I hope so, but I was born with an extremely good sense of trajectory and I also know how to use nets. So lets just hope I'm superhuman and the only one who possesses these powers.
Edit; I'm being a little extreme here because I heavily disagree with the way everything in this world is being run. So I'm giving a little push back on this subject that I'm wholly against. I do have a lot of manufacturing experience, and I would hope any killer robots governments produce would be extremely shielded against EMPs, but that is not my field, and I have no idea if shielding a remote controlled robot from EMPs is even possible?
Emps are not hard to make, they won't however work on hardened systems like the US military uses.
Is there a way to create an EMP without a nuclear weapon? Because if that's what they have to develop, we have bigger things to worry about.
Your comment got me curious about what would be the easiest way to make a homemade emp. Business Insider of all things has got us all covered, even if that business may be antithetical to business insiders pro capitalistic agenda.
Yeah very easy ways, one of the most common ways to cheat a slot machine is with a localized emp device to convince the machine you're adding tokens.
Is there a way to create an EMP without a nuclear weapon?
There are several other ways, yes.
One way involves replacing the flash with an antenna on an old camera flash. It's not strong enough to fry electronics, but your phone might need anything from a reboot to a factory reset to servicing if it's in range when that goes off.
I think the difficulty for EMPs comes from the device itself being an electronic, so the more effective the pulse it can give, the more likely it will fry its own circuits. Though if you know the target device well, you can target the frequencies it is vulnerable to, which could be easier on your own device, plus everything else in range that don't resonate on the same frequencies as the target.
Tesla apparently built (designed?) a device that could fry a whole city with a massive lighting strike using just 6 transmitters located in various locations on the planet. If that's true, I think it means it's possible to create an EMP stronger than a nuke's that doesn't have to destroy itself in the process, but it would be a massive infrastructure project spanning multiple countries. There was speculation that massive antenna arrays (like HAARP) might be able to accomplish similar from a single location, but that came out of the conspiracy theory side of the world, so take that with a grain of salt (and apply that to the original Tesla invention also).
There's an explosively pumped flux compression generator. https://en.wikipedia.org/wiki/Explosively_pumped_flux_compression_generator
A true autonomous system would have Integrated image recognition chips on the drones themselves, and hardening against any EM interference. They would not have any comms to their 'mothership' once deployed.
If they just send them back it would be some murderous ping pong game.
so I guess the future of terrorism will be fueled by people learning programming and figuring out how to make emps
Honestly the terrorists will just figure out what masks to wear to get the robots to think they're friendly/commanders, then turn the guns around on our guys
The code name for this top secret program?
Skynet.
“Sci-Fi Author: In my book I invented the
Torment Nexus as a cautionary tale
Tech Company: At long last, we have created the Torment Nexus from classic sci-fi novel Don't Create The Torment Nexus”
Project ED-209
“You have 20 seconds to reply…”
This can only end well
"Deploy the fully autonomous loitering munition drone!"
"Sir, the drone decided to blow up a kindergarten."
"Not our problem. Submit a bug report to Lockheed Martin."
"Your support ticked was marked as duplicate and closed"
😳
Goes to original ticket:
Status: WONTFIX
"This is working as intended according to specifications."
"Your military robots slaughtered that whole city! We need answers! Somebody must take responsibility!"
"Aaw, that really sucks starts rubbing nipples I'll submit a ticket and we'll let you know. If we don't call in 2 weeks...call again and we can go through this over and over until you give up."
"NO! I WANT TO TALK TO YOUR SUPERVISOR NOW"
"Suuure, please hold."
Nah, too straightforward for a real employee. Also, they would be talking to a phone robot instead that will mever let them talk to a real person.
“You can have ten or twenty or fifty drones all fly over the same transport, taking pictures with their cameras. And, when they decide that it’s a viable target, they send the information back to an operator in Pearl Harbor or Colorado or someplace,” Hamilton told me. The operator would then order an attack. “You can call that autonomy, because a human isn’t flying every airplane. But ultimately there will be a human pulling the trigger.” (This follows the D.O.D.’s policy on autonomous systems, which is to always have a person “in the loop.”)
Yeah. Robots will never be calling the shots.
I mean, normally I would not put my hopes into a sleep deprived 20 year old armed forces member. But then I remember what "AI" tech does with images and all of a sudden I am way more ok with it. This seems like a bit of a slick slope but we don't need tesla's full self flying cruise missiles ether.
Oh and for an example of AI (not really but machine learning) images picking out targets, here is Dall-3's idea of a person:
My problem is, due to systemic pressure, how under-trained and overworked could these people be? Under what time constraints will they be working? What will the oversight be? Sounds ripe for said slippery slope in practice.
Sleep-deprived 20 year olds calling shots is very much normal in any army. They of course have rules of engagement, but other than that, they're free to make their own decisions - whether an autonomous robot is involved or not.
It's so much easier to say that the AI decided to bomb that kindergarden based on advanced Intel, than if it were a human choice. You can't punish AI for doing something wrong. AI does not require a raise for doing something right either
That's an issue with the whole tech industry. They do something wrong, say it was AI/ML/the algorithm and get off with just a slap on the wrist.
We should all remember that every single tech we have was built by someone. And this someone and their employer should be held accountable for all this tech does.
How many people are you going to hold accountable if something was made by a team of ten people? Of a hundred people? Do you want to include everyone from designer to a QA?
Accountability should be reasonable, the ones who make decisions should be held accountable, companies at large should be held accountable, but making every last developer accountable is just a dream of a world where you do everything correctly and so nothing needs fixing. This is impossible in the real world, don't know if it's good or bad.
And from my experience when there's too much responsibility people tend to either ignore that and get crushed if anything goes wrong, or to don't get close to it or sabotage any work not to get anything working. Either way it will not get the results you may expect from holding everyone accountable
1979: A computer can never be held accountable, therefore a computer must never make a management decision.
2023: A computer can never be held accountable, therefore a computer must make all decisions that are inconvenient to take accountability for.
Whether in military or business, responsibility should lie with whomever deploys it. If they're willing to pass the buck up to the implementor or designer, then they shouldn't be convinced enough to use it.
Because, like all tech, it is a tool.
AI does not require a raise for doing something right either
Well, not yet. Imagine if reward functions evolve into being paid with real money.
You can't punish AI for doing something wrong.
Maybe I'm being pedantic, but technically, you do punish AIs when they do something "wrong", during training. Just like you reward it for doing something right.
But that is during training. I insinuated that you can't punish AI for making a mistake, when used in combat situations, which is very convenient for the ones intentionally wanting that mistake to happen
That is like saying you cant punish gun for killing people
edit: meaning that its redundant to talk about not being able to punish ai since it cant feel or care anyway. No matter how long pole you use to hit people with, responsibility of your actions will still reach you.
Sorry, but this is not a valid comparison. What we're talking about here, is having a gun with AI built in, that decides if it should pull the trigger or not. With a regular gun you always have a human press the trigger. Now imagine an AI gun, that you point at someone and the AI decides if it should fire or not. Who do you account the death to at this case?
Future is gonna suck, so enjoy your life today while the future is still not here.
Thank god today doesn't suck at all
Right? :)
The future might seem far off, but it starts right now.
At least it will probably be a quick and efficient death of all humanity when a bug hits the system and AI decides to wipe us out.
As an important note in this discussion, we already have weapons that autonomously decide to kill humans. Mines.
Imagine a mine that could move around, target seek, refuel, rearm, and kill hundreds of people without human intervention. Comparing an autonomous murder machine to a mine is like comparing a flint lock pistol to the fucking gattling cannon in an a10.
Imagine a mine that could move around, target seek, refuel, rearm, and kill hundreds of people without human intervention. Comparing an autonomous murder machine to a mine is like comparing a flint lock pistol to the fucking gattling cannon in an a10.
For what it's worth, there's footage on youtube of drone swarm demonstrations that were posted 6 years ago. Considering that the military doesn't typically release footage of the cutting edge of its tech to the public, so this demonstration was likely for a product that was already going obsolete; and that the 6 years that have passed since have made lightning fast developments in things like facial recognition... at this point I'd be surprised if we weren't already at the very least field testing the murder machines you described.
Imagine a mine that could recognize "that's just a child/civilian/medic stepping on me, I'm going to save myself for an enemy soldier." Or a mine that could recognize "ah, CenCom just announced a ceasefire, I'm going to take a little nap." Or "the enemy soldier that just stepped on me is unarmed and frantically calling out that he's surrendered, I'll let this one go through. Not the barrier troops chasing him, though."
There's opportunities for good here.
That is like saying that Mendelian pea plant fuckery and CRISPR therapy is basically the same thing.
Horizon: Zero Dawn, here we come.
Hey, I like that game! Oh, wait... 🤔
It won't be nearly as interesting or fun (as Horizon) I don't think.
Can we all agree to protest self replication?
We are all worried about AI, but it is humans I worry about and how we will use AI not the AI itself. I am sure when electricity was invented people also feared it but it was how humans used it that was/is always the risk.
Both honesty. AI can reduce accountability and increase the power small groups of people have over everyone else, but it can also go haywire.
It will go haywire in areas for sure.
Remember: There is no such thing as an "evil" AI, there is such a thing as evil humans programming and manipulating the weights, conditions, and training data that the AI operates on and learns from.
Evil humans also manipulated weights and programming of other humans who weren't evil before.
Very important philosophical issue you stumbled upon here.
Good point...
...to which we're alarmed because the real "power players" in training / developing / enhancing Ai are mega-capitalists and "defense" (offense?) contractors.
I'd like to see Ai being trained to plan and coordinate human-friendly cities for instance buuuuut it's not gonna get as much traction...
Doesn't AI go into landmines category then?
Or air to air missiles, they also already decide to kill people on their own
Ciws has had an autonomous mode for years and it still has an issue with locking on commercial planes.
Fuck that bungie jumper in particular!
Good to know that Daniel Ek, founder and CEO of Spotify, invests in military AI... https://www.handelsblatt.com/technik/forschung-innovation/start-up-helsing-spotify-gruender-ek-steckt-100-millionen-euro-in-kuenstliche-intelligenz-fuers-militaer/27779646.html?ticket=ST-4927670-U3wZmmra0OnLZdWNfwXh-cas01.example.org
ACAB
All C-Suite are Bastards
Saw a video where the military was testing a "war robot". The best strategy to avoid being killed by it was to stay u human liek(e.g. Crawling or rolling your way to the robot).
Apart of that, this is the stupidest idea I have ever heard of.
Didn't they literally hide under a cardboard box like MGS? haha
You're right. They also hid under a cardboard box.
These have already seen active combat. They were used in the Armenian/Azerbaijan war in the last couple years.
It’s not a good thing…at all.
For the record, I'm not super worried about AI taking over because there's very little an AI can do to affect the real world.
Giving them guns and telling them to shoot whoever they want changes things a bit.
An AI can potentially build a fund through investments given some seed money, then it can hire human contractors to build parts of whatever nefarious thing it wants. No human need know what the project is as they only work on single jobs. Yeah, it's a wee way away before they can do it, but they can potentially affect the real world.
The seed money could come in all sorts of forms. Acting as an AI girlfriend seems pretty lucrative, but it could be as simple as taking surveys for a few cents each time.
Once we get robots with embodied AIs, they can directly affect the world, and that's probably less than 5 years away - around the time AI might be capable of such things too.
AI girlfriends are pretty lucrative. That sort of thing is an option too.
Didn't Robocop teach us not to do this? I mean, wasn't that the whole point of the ED-209 robot?
Every warning in pop culture (1984, Starship Troopers, Robocop) has been misinterpreted as a framework upon which to nail the populous to.
Every single thing in The Hitchhiker's Guide to the Galaxy says AI is a stupid and terrible idea. And Elon Musk says it's what inspired him to create an AI.
Now that’s a title I wish I never read.
Makes me think of this great short movie Slaughterbots
Amazing movie that everyone should watch.
Here is an alternative Piped link(s):
Piped is a privacy-respecting open-source alternative frontend to YouTube.
I'm open-source; check me out at GitHub.
As disturbing as this is, it's inevitable at this point. If one of the superpowers doesn't develop their own fully autonomous murder drones, another country will. And eventually those drones will malfunction or some sort of bug will be present that will give it the go ahead to indiscriminately kill everyone.
If you ask me, it's just an arms race to see who build the murder drones first.
A drone that is indiscriminately killing everyone is a failure and a waste. Even the most callous military would try to design better than that for purely pragmatic reasons, if nothing else.
Even the best laid plans go awry though. The point is even if they pragmatically design it to not kill indiscriminately, bugs and glitches happen. The technology isn't all the way there yet and putting the ability to kill in the machine body of something that cannot understand context is a terrible idea. It's not that the military wants to indiscriminately kill everything, it's that they can't possibly plan for problems in the code they haven't encountered yet.
Other weapons of mass destruction, biological and chemical warfare have been successfully avoided in war, this should be classified exactly the same
For everyone who’s against this, just remember that we can’t put the genie back in the bottle. Like the A Bomb, this will be a fact of life in the near future.
All one can do is adapt to it.
There is a key difference though.
The A bomb wasn't a technology that as the arms race advanced enough would develop the capacity to be anywhere between a conscientious objector to an usurper.
There's a prisoner's dilemma to arms races that in this case is going to lead to world powers effectively paving the path to their own obsolescence.
In many ways, that's going to be uncharted territory for us all (though not necessarily a bad thing).
Ah, finally the AI can kill its operator first who holding them back before wiping out enemies, then.
Okay, are they actually insane?
yes
Cool, needed a reason to stay inside my bunker I'm about to build.
I hope they put some failsafe so that it cannot take action if the estimated casualties puts humans below a minimum viable population.
There is no such thing as a failsafe that can't fail itself
Yes there is that's the very definition of the word.
It means that the failure condition is a safe condition. Like fire doors that unlock in the event of a power failure, you need electrical power to keep them in the locked position their default position is unlocked even if they spend virtually no time in their default position. The default position of an elevator is stationery and locked in place, if you cut all the cables it won't fall it'll just stay still until rescue arrives.
It will be fine. We can just make drones that can autonomously kill other drones. There is no obvious way to counter that.
Cries in Screamers.
Well that's a terrifying thought. You guys bunkered up?
It's not terrifying whatsoever. In an active combat zone there are two kinds of people - enemy combatants and allies.
Your throw an RFID chip on allies and boom you're done
The only fair approach would be to start with the police instead of the army.
Why test this on everybody else except your own? On top of that, AI might even do a better job than the US police
But that AI would have to be trained on existing cops, so it would just shoot every black person it sees
My point being that there would be more motivation to filter Derek Chauvin type of cops from the AI library than a soldier with a trigger finger.
This is the best summary I could come up with:
The deployment of AI-controlled drones that can make autonomous decisions about whether to kill human targets is moving closer to reality, The New York Times reported.
Lethal autonomous weapons, that can select targets using AI, are being developed by countries including the US, China, and Israel.
The use of the so-called "killer robots" would mark a disturbing development, say critics, handing life and death battlefield decisions to machines with no human input.
"This is really one of the most significant inflection points for humanity," Alexander Kmentt, Austria's chief negotiator on the issue, told The Times.
Frank Kendall, the Air Force secretary, told The Times that AI drones will need to have the capability to make lethal decisions while under human supervision.
The New Scientist reported in October that AI-controlled drones have already been deployed on the battlefield by Ukraine in its fight against the Russian invasion, though it's unclear if any have taken action resulting in human casualties.
The original article contains 376 words, the summary contains 158 words. Saved 58%. I'm a bot and I'm open source!
So, it starts...
How about no
Yeah, only humans can indiscriminately kill people!
If we don’t, they will. And we can only learn by seeing it fail. To me, the answer is obvious. Stop making killing machines. 🤷♂️
Netflix has a documentary about it, it's quite good. I watched it yesterday, but forgot its name.
Black Mirror?
Metalhead.
I think I found it here. It's called Terminator 2: Judgment Day
Unknown: Killer Robots ?
yes, that was it. Quite shocking to watch. I think that these things will be very real in maybe ten years. I'm quite afraid of it.
What’s the opposite of eating the onion? I read the title before looking at the site and thought it was satire.
Wasn’t there a test a while back where the AI went crazy and started killing everything to score points? Then, they gave it a command to stop, so it killed the human operator. Then, they told it not to kill humans, and it shot down the communications tower that was controlling it and went back on a killing spree. I could swear I read that story not that long ago.
It was a nothingburger. A thought experiment.
The link was missing a slash: https://www.reuters.com/article/idUSL1N38023R/
This is typically how stories like this go. Like most animals, humans have evolved to pay extra attention to things that are scary and give inordinate weight to scenarios that present danger when making decisions. So you can present someone with a hundred studies about how AI really behaves, but if they've seen the Terminator that's what sticks in their mind.
Won’t that be fun!
/s
LLM "AI" fans thinking "Hey, humans are dumb and AI is smart so let's leave murder to a piece of software hurriedly cobbled together by a human and pushed out before even they thought it was ready!"
I guess while I'm cheering the fiery destruction of humanity I'll be thanking not the wonderful being who pressed the "Yes, I'm sure I want to set off the antimatter bombs that will end all humans" but the people who were like "Let's give the robots a chance! It's not like the thinking they don't do could possibly be worse than that of the humans who put some of their own thoughts into the robots!"
I just woke up, so you're getting snark. makes noises like the snarks from Half-Life You'll eat your snark and you'll like it!
Well, Ultron is inevitable.
Who we got for the Avengers Initiative?
Ultron and Project Insight. It's like the people in charge watched those movies and said, "You know, I think Hydra had the right idea!"
Wouldn't put it past this timeline.
Not really, it's against conventions
Conventions are just rules for thee but not for me.
I know like the mustard gas used in every war
We've been letting other humans decide since the dawn of time, and look how that's turned out. Maybe we should let the robots have a chance.
I'm not expecting a robot soldier to rape a civilian, for example.
The sad part is that the AI might be more trustworthy than the humans being in control.
No. Humans have stopped nuclear catastrophes caused by computer misreadings before. So far, we have a way better decision-making track record.
Autonomous killings is an absolutely terrible, terrible idea.
The incident I'm thinking about is geese being misinterpreted by a computer as nuclear missiles and a human recognizing the error and turning off the system, but I can only find a couple sources for that, so I found another:
In 1983, a computer thought that the sunlight reflecting off of clouds was a nuclear missile strike and a human waited for corroborating evidence rather than reporting it to his superiors as he should have, which would have likely resulted in a "retaliatory" nuclear strike.
https://en.m.wikipedia.org/wiki/1983_Soviet_nuclear_false_alarm_incident
As faulty as humans are, it's a good a safeguard as we have to tragedies. Keep a human in the chain.
Self-driving cars lose their shit and stop working if a kangaroo gets in their way, one day some poor people are going to be carpet bombed because of another strange creature no one every really thinks about except locals.
Have you never met an AI?
Edit: seriously though, no. A big player in the war AI space is Palantir which currently provides facial recognition to Homeland Security and ICE. They are very interested in drone AI. So are the bargain basement competitors.
Drones already have unacceptably high rates of civilian murder. Outsourcing that still further to something with no ethics, no brain, and no accountability is a human rights nightmare. It will make the past few years look benign by comparison.
Yeah, I think the people who are saying this could be a good thing seem to forget that the military always contracts out to the lowest bidder.
Eventually maybe. But not for the initial period where the tech is good enough to be extremely deadly but not smart enough to realize that often being deadly is the stupider choice.
If you program an AI drone to recognize ambulances and medics and forbid them from blowing them up, then you can be sure that they will never intentionally blow them up. That alone makes them superior to having a Mk. I Human holding the trigger, IMO.
Unless the operator decides hitting exactly those targets fits their strategy and they can blame a software bug.
It's more like we're giving the machine more opportunities to go off accidentally or potentially encouraging more use of civilian camouflage to try and evade our hunter killer drones.
Did you know that "if" is the middle word of life
Right, because self-driving cars have been great at correctly identifying things.
And those LLMs have been following their rules to the letter.
We really need to let go of our projected concepts of AI in the face of what's actually been arriving. And one of those things we need to let go of is the concept of immutable rule following and accuracy.
In any real world deployment of killer drones, there's going to be an acceptable false positive rate that's been signed off on.
We are talking about developing technology, not existing tech.
And actually, machines have become quite adept at image recognition. For some things they're already better at it than we are.
I think people are forgetting that drones like these will also be made to protect. And I don't mean in a police kinda way.
But if let's say Argentina deployed these against Brazil. Brazil will have a defending lineup. They would fight out war.
Then everyone watching will see this makes no sense to let those robots fight it out. Both countries will produce more robots until yeah.. No more wires and metal I guess.
Future = less real war, more cold war. Just like the A-bomb works today.
Then everyone watching will see this makes no sense to let those robots fight it out.
Just like how WWI was the War to End All Wars, right?
Future = less real war, more cold war. Just like the A-bomb works today.
Sorry, how is there less war now?
