Russia's National Anti-Terrorism Committee (NAC) said the special services had confiscated arms, ammunition and explosives - including surface-to-air missiles and a flamethrower - in the breakaway Abkhazia region of Georgia, the South Caucasus country with which Russia went to war in 2008.
Any security breach around the games could be embarrassing for President Vladimir Putin, who has taken a personal interest in the event and hopes it can be used to boost Russia's image.
Abkhazia, which Russia recognised as an independent nation after the war, is adjacent to Sochi on the Black Sea coast. The assailants had also planned attacks in the runup to the games in February 2014, it said.
"Russia's FSB (security service) was able to establish that the fighters planned to move the weapons to Sochi from 2012 to 2014 and use them to carry out terrorist acts before and during the Olympic Games," NAC said in a statement.
The NAC blamed the plot on the Caucasus Emirate, one of the leading groups in an insurgency against Russian rule in the volatile North Caucasus, where Russian troops have fought two wars in Chechnya since the collapse of the Soviet Union in 1991.
It suggested the group's leader, Doku Umarov, had been cooperating with Georgian special services but did not give any details to support this allegation and the NAC report could not be independently verified.
Russia has often said Georgia may be plotting further aggression following the war, an accusation Georgia rejects.
NAC said the weapons included Igla and Strela portable surface-to-air missiles, two anti-tank guided missiles and 36 mortar bombs as well as a flame thrower, grenade launchers, explosive devices, and anti-tank and anti-personnel mines.
The insurgents want to create an Islamic state and say they were behind a suicide bombing at Moscow's Domodedovo airport that killed 37 people in January 2011 and twin bombings that killed 40 people on Moscow's metro in 2010.
One group, the Muslim Circassians, driven from their homes in the North Caucasus by Russian soldiers in the mid-19th century, say the Games are being held on a "mass grave".
They say 1.5 million of their predecessors perished as Russian embarked on a mass expulsion of their people to try to conquer the Caucasus.
The NAC did not say how the Caucasus Emirate might have got hold of the arms in Abkhazia, but many could have been available following the five-day war in 2008 between Georgia and Russia over Abkhazia and South Ossetia, another separatist region.
The International Olympic Committee has said it is confident Russia will provide a safe Games and Russia is working hard to prevent its image being tarnished by security problems at an events that will be watched around the world.
Putin, a sports lover who likes to ski in Sochi and has an official residence there, has staked a lot on the Olympics, and staging a successful Games is a matter of national pride.
Russia's 2014 Winter Games chief, Dmitry Chernyshenko, said on Sunday Sochi was ready to step into the spotlight and underlined Putin's personal involvement.
"Since the start of the bidding campaign he was personally and actively involved in everything," Chernyshenko said.
"He personally guaranteed the strong commitments which have been delivered. Delivery of the first Winter Games in the history of our country. It was his baby. That is why we call him the captain of our team."