In an open letter to the UN, they say lethal autonomous weapons threaten a "third revolution in warfare".
More than 100 robotics and artificial intelligence (AI) industry leaders signed the letter.
Mr Musk has long spoken out about the dangers of artificial intelligence.
Fears about the rise of AI and the risk posed by machines have escalated in recent years as the technology's use in warfare develops.
The letter to the UN Convention on Certain Conventional Weapons calls for solutions to "prevent an arms race" in lethal autonomous weapons.
These future weapons - dubbed "killer robots" - are machines programmed to hit people or targets, which operate autonomously on the battlefield.
"Once developed, they will permit armed conflict to be fought at a scale greater than ever, and at timescales faster than humans can comprehend," the letter states.
"These can be weapons of terror, weapons that despots and terrorists use against innocent populations, and weapons hacked to behave in undesirable ways," it said.
"We do not have long to act. Once this Pandora's box is opened, it will be hard to close," they added.
The letter was signed by 116 robotics and artificial intelligence leaders from around the world, including Mr Musk and Google's DeepMind co-founder Mustafa Suleyman.
A UN group examining autonomous weapons was due to meet on Monday and is now expected to meet in November.
The letter was released at an international AI conference in Melbourne on Monday.
A similar letter warning of the dangers of autonomous weapons, signed by a slew of tech experts including Stephen Hawking, Apple co-founder Steve Wozniak and Mr Musk, was released in 2015.
Mr Musk has long been a vocal critic about the dangers of AI.
The billionaire founder of Space X has described artificial intelligence as humanity's "biggest existential threat".
He founded OpenAI, a non-profit company working for "safer artificial intelligence".
Musk recently clashed with another tech leader, Facebook founder Mark Zuckerberg, who is less fearful about the future of AI.
It comes after the Facebook boss said that the doomsday scenario put forward by Mr Musk was unhelpful.
Mr Musk tweeted: "I've talked to Mark about this. His understanding of the subject is limited."