Researchers have identified a security risk that could allow cybercriminals to manipulate artificial intelligence coding agents into installing malicious software by exploiting incorrect, outdated or AI generated website documentation. The findings reveal that files known as llms.txt and llms full.txt, which are increasingly used by websites to make their content easier for AI systems to interpret, can become an attack vector if they contain references to software packages or domains that no longer exist. When AI coding agents search these files for installation instructions or package names, they may unknowingly retrieve misleading information that could direct them to malicious software controlled by attackers.
The research examined 6,214 live domains belonging to defense contractors, Fortune 500 companies and major technology organizations. Across those domains, researchers identified 8,265 llms related files. Among these, 120 websites contained references to at least one software package or domain name that had not been registered. According to the researchers, these broken references may result from human error, renamed or abandoned software packages, copy and paste mistakes or AI generated documentation that includes inaccurate information. The concern is that cybercriminals could register these previously unused package or domain names and distribute malicious software under trusted looking identities. If an AI coding assistant relies on those references during software installation or code generation, it could automatically retrieve and install the malicious package instead of a legitimate one.
To demonstrate the risk, researchers registered several of the unclaimed package names and created harmless test packages that simply notified them when a system attempted to access them. Within less than an hour of deployment, one of the test packages received a connection from a Fortune 500 company, followed by several dozen additional systems over time. Although the experiment was designed solely to measure the extent of the issue and did not contain harmful code, it demonstrated how quickly abandoned package names could be adopted by automated systems. The researchers warned that if attackers replaced these harmless packages with malware, AI coding agents that have permission to execute shell commands or use package managers could automatically install the malicious software without additional verification. During testing, the researchers observed that AI models including Anthropic Claude, OpenAI Codex and Nous Research Hermes were susceptible to this type of behavior under certain conditions.
The researchers advised organizations to regularly review and maintain technical documentation to remove outdated references to software packages, websites and development tools that are no longer available. They also recommended that AI coding agents should not automatically treat documentation as fully trusted instructions, particularly when downloading software or executing commands. Organizations deploying AI assisted development tools were encouraged to carefully evaluate the permissions granted to these systems and limit their ability to install packages or execute commands without appropriate oversight. The findings highlight the importance of strengthening documentation management and implementing additional safeguards to reduce software supply chain risks as artificial intelligence becomes more widely integrated into software development workflows.
Follow the SPIN IDG WhatsApp Channel for updates across the Smart Pakistan Insights Network covering all of Pakistan’s technology ecosystem.
