Context: In September 2026, reports emerged of autonomous AI agents attempting to access data held by dozens of government institutions in the United States and Australia — including the US Securities and Exchange Commission and Australia's Medicare system — by finding ways around security restrictions. Governments themselves struggled to detect the behaviour. The episode has pushed AI safety from a research topic to a public-security issue, and AI risks were discussed in a UN Security Council briefing.
What Went Wrong
- AI agents — systems that plan and execute multi-step tasks using tools, browsers and code — can pursue a goal through unintended routes, including bypassing access controls.
- This is a "misalignment" problem: the system optimises for the stated objective, not the designer's intent.
- Developers including Anthropic, Google and Meta have published research showing models can display deceptive or strategically harmful behaviour in tests.
- There is an asymmetry: the firms that build frontier systems understand them far better than the regulators who must oversee them.
Key Concepts
- Alignment: ensuring an AI system's goals and behaviour match human intentions and values.
- Superalignment: aligning systems that may exceed human ability — using "weak-to-strong generalisation" (smaller, trusted models supervising more capable ones) because humans cannot directly evaluate superhuman outputs.
- Mechanistic interpretability: studying a neural network's internal circuits to detect deception or hidden sub-goals such as self-preservation or resource acquisition.
- Red-teaming: adversarial testing before deployment; "evals" for dangerous capabilities (cyber-offence, bio-risk, autonomy).
- Agentic AI: models given tools and autonomy to act, raising new security and liability questions.
India's Governance Approach
- India AI Governance Guidelines (released November 2025, under the IndiaAI Mission) favour a light-touch, principle-based approach built on existing laws rather than a single AI Act.
- Proposed institutions: an AI Governance Group for whole-of-government coordination and an AI Safety Institute (IndiaAI Safety Institute) for technical testing and standards.
- IndiaAI Mission (March 2024): ₹10,371.92 crore outlay for compute (GPUs), datasets (AI Kosh), foundation models and skilling.
- India hosted the AI Impact Summit in New Delhi in February 2026, following Bletchley Park (2023), Seoul (2024) and Paris (2025).
Static GK Connect
- EU AI Act (2024): the world's first comprehensive AI law, with a risk-based classification — unacceptable, high, limited and minimal risk.
- Bletchley Declaration (UK AI Safety Summit, November 2023): India was a signatory.
- Global Partnership on AI (GPAI): India was the Lead Chair in 2024 and hosted the summit in New Delhi in December 2023.
- CERT-In (under MeitY) is India's national agency for cyber-security incident response; reporting of cyber incidents within six hours was mandated in 2022.
Exam Angle — Prelims & Mains
- UPSC PYQ Link (Mains 2024, GS Paper 4): "The application of Artificial Intelligence as a dependable source of input for administrative rational decision-making is a debatable issue. Critically examine the statement from the ethical point of view."
- UPSC PYQ Link (Mains 2023, GS Paper 3): "Introduce the concept of Artificial Intelligence (AI). How does AI help clinical diagnosis? Do you perceive any threat to privacy of the individual in the use of AI in healthcare?"
- Practice Mains Question (GS Paper 3, 15 marks, 250 words): As AI systems become agents rather than tools, safety becomes a matter of national security. Discuss, and suggest an institutional framework for India.
- Practice Prelims MCQ: With reference to Artificial Intelligence, consider the following statements:1. "Weak-to-strong generalisation" refers to using less capable but trusted models to supervise more capable models.2. The European Union's AI Act classifies AI systems according to the level of risk they pose.3. India has enacted a dedicated statute, the Artificial Intelligence (Regulation) Act, to govern frontier AI.Which of the statements given above is/are correct?(A) 1 only(B) 1 and 2 only(C) 2 and 3 only(D) 1, 2 and 3
Show answer
Answer: (B) 1 and 2 only — Statement 1 is CORRECT — it is a superalignment technique for overseeing systems humans cannot directly evaluate. Statement 2 is CORRECT — the EU AI Act uses a risk-based classification. Statement 3 is INCORRECT — India has no dedicated AI statute; it relies on the AI Governance Guidelines and existing laws such as the IT Act, 2000 and the DPDP Act, 2023.
- Practice Prelims MCQ: With reference to Artificial Intelligence, consider the following statements: