The approaching US withdrawal from Afghanistan is not an honourable retreat – it’s a capitulation. The best the Americans can hope for in exit talks with the Taliban, due to resume in Doha later this month, is a promise that coalition troops, unlike the British army led by General Elphinstone in 1842, will not depart under fire. After more than 17 years of conflict, with at least 38,000 civilians killed and millions more injured, traumatised or exiled, none of the long-term objectives set out by George W Bush following the 2001 invasion has been met. In short, the US has lost the war and lost badly.

The al-Qaida terrorists who used Afghanistan as a base from which to launch the 9/11 attacks have not been wholly vanquished, as Bush promised. Their former leader, Osama bin Laden, is dead but the group, and likewise Islamic State, made territorial gains in Afghanistan last year, according to UN experts. It is unlikely that Taliban leaders could in future prevent jihadists once again using parts of the country as a terrorist safe haven – a key demand of American negotiators – even if they sincerely wanted to.

The idea, promoted by successive US administrations and Nato partners such as Britain, that Afghanistan could become a model nation-building exercise has long since been exposed as a neoliberal fantasy. This is not to dismiss the tenacious efforts of British and allied forces on the ground who struggled valiantly, for instance, to bring stability to Helmand province. But they, and the Afghan people have paid a terrible price for a lack of clarity and candour on the part of the politicians who sent them there.

Another delusion – that Afghan security forces could be trained and equipped to a point where they could, unaided, contain the Taliban and control the country – has also been shattered. Despite Nato’s best efforts, 30 to 40 Afghan soldiers and police were being killed each day last autumn. About 45,000 Afghan soldiers have died since 2014. This increase in mortality mirrors the decrease in western troop numbers since the 2011 peak.

Despite massive diplomatic and developmental assistance, the Afghan government holds limited sway beyond the barricaded safe zones of central Kabul. The Taliban refuse to allow it to participate in withdrawal talks, dismissing it as a mere puppet. The implications, once the Americans have gone, are ominous. A rapid descent into civil war, involving government forces, jihadist groups and rival warlords, in a rerun of not-forgotten 1990s anarchy, is a strong possibility. Last year saw record civilian deaths, caused by terror bombings, intensified fighting and increased US airstrikes. Coming years may prove yet more deadly.

Even if the worst is avoided, Afghan civil society organisations and education, public health and women’s rights advocates rightly dread the inclusion of Taliban Sunni fundamentalists in any postwar political settlement. Despite assurances to the contrary, the prospective return of Taliban hardliners to positions of influence presages a new dark age of repression, discrimination and bigotry. And how long will any pretence of democratic government persist? As in the past, Afghanistan could quickly revert to a Great Game free-for-all involving Pakistan, India, Russia, Iran – and now China, too. In sum, the Americans are leaving an unholy mess that they, more than any other individual actor, helped create.

From a narrow US viewpoint, there are still plenty of reasons for quitting while they’re behind. Donald Trump campaigned against “endless wars”, and recently decided to halve the current US force of 14,000. Most voters want the troops out. Billions of dollars have been expended to no lasting effect, while millions more have been lost to corruption. In the absence of any post-9/11 foreign terrorist attacks on US soil, Bush’s old claim that fighting terror abroad avoids the need to fight it at home carries diminishing weight.

In a significant shift, the New York Times which, like all major US media, supported the 2001 invasion, called this week for the troops to come home – and eviscerated the overall US approach to fighting terror. Far from eradicating international terrorism as Bush pledged, the “global war on terror” was in large part responsible for the worldwide growth since 2001 of Islamist-inspired terrorist groups, the paper’s editorial noted.

Put this another way. The British-supported US global war on terror, which has claimed about half a million lives, been waged across 80 nations, corroded respect for human rights and international law, and cost an estimated $5.9tn, was, from the outset, a catastrophic mistake based on a false premise, aggravated by self-righteous arrogance and an unforgivable ignorance of the world beyond America’s shores. At long last, reality dawns!

Yet wait. Is this a lesson the US leviathan in all its many guises can now safely be assumed to have learned? Not really. Trump’s announcement this week that he plans to redeploy troops and air power from Syria to neighbouring Iraq, in order to “keep watch” on Iran, is the latest clue to Washington’s next Middle East war of choice.

US intelligence chiefs told Trump last week that Iran was not a pressing concern, noting that it continued (unlike the US) to honour the 2015 nuclear pact. But Trump and his gang of ideologues and know-nothings, egged on by Israel and the Saudis, persist in mendaciously characterising the Tehran regime as the “world’s leading state sponsor of terror”, and a direct threat to US national security. That is what they said about Saddam Hussein. In Washington speak, it’s a casus belli.

Perhaps there is something in the nature and psyche of the ever-embattled American republic that means it needs monstrous foreign enemies – hateful bogeymen such as Saddam, Manuel Noriega, Muammar Gaddafi, Nicolás Maduro or Ayatollah Ali Khamenei – in order to sustain its own founding myths. King George, the original “tyrant”, was vanquished by force of arms, and they have never forgotten it. It’s a mindset that has to change.

Instead of threatening to begin a new war with Iran, US leaders should be trying much harder to end the old one in Afghanistan in a responsible, sustainable, humane manner. Don’t just scuttle away. The US should work with all elements of Afghan society, including Taliban moderates – in a civilian, not military, capacity – to reconstruct the country the war destroyed. After 17 or so years of ultimately pointless, criminal mayhem, an apology plus reparations to the Afghan people would be a good start.