They keep their heads down and melt into the crowd. They know they're the lucky ones

Despite the best efforts of the Pakistan government to seal the border from the tidal wave of the hungry and frightened on the other side, tens of thousands of Afghans have poured into the North-West Frontier Province over the past fortnight. They are not to be seen in the established camps, for they have melted into tribal Pathan society and are keeping their heads down to avoid detection and expulsion.

Despite the best efforts of the Pakistan government to seal the border from the tidal wave of the hungry and frightened on the other side, tens of thousands of Afghans have poured into the North-West Frontier Province over the past fortnight. They are not to be seen in the established camps, for they have melted into tribal Pathan society and are keeping their heads down to avoid detection and expulsion.

But go looking for them and you find them everywhere. They tell desperate stories. And they also tell about the unlucky millions, trapped on the other side now that Pakistan has imposed strict controls along the 870-mile border.

The North-West Frontier Province is Pakistan's safety valve and buffer zone, a political airlock between the teeming, fertile plains of the Punjab and the rocky hills and incessant turmoil of Afghanistan. Most people here are tribal Pathans, ethnically the same as the majority population of Afghanistan, and the scenery, too, is distinctly Afghan. The contrast is stark: cross the thunderous, rusting box-girder bridge over the Indus and you are out of the Indian subcontinent and into central Asia: a land of harsh, treeless hills with jagged spines, dry, meagre fields, dusty towns full of houses with blank, forbidding walls of dried mud. "This is a very terrible scene," said my Punjabi driver as we left the fields of his homeland behind. "When we Pakistanis come here we think it is very terrible."

We were trying to get up to the town of Parachinar, a few miles from the Afghan border, where, according to people we had spoken to further east, the refugees were streaming in. But the so-called "tribal agencies" along the border are closed to foreigners who do not have special permission. In normal times most things in Pakistan can be fixed over a cup of tea, but the times are far from normal. The sign on the fort-like entrance gate to the Kurram Agency said Kush-am-deed, "Welcome", but that did not extend to me. After a long wait, permission to enter was refused and we were sent back to the nearest town under armed guard. The Kurram Scout, cradling a Kalashnikov, told us: "One thousand refugees came in through Parachinar but we arrested them and pushed them back. Security is very tight at the border now and we are letting nobody in." But even though we had to give up the idea of getting to the border, newly arrived refugees were not hard to find. We looked around for somewhere to eat a late breakfast and found a grimy stew joint next to a petrol station. Mopping up the lentils and curried goat, we asked the waiter if he had come across any freshly arrived refugees. "The four of us here," he said, indicating the other men working in the café, "came through one week back. We came over the hard way, crawling through the undergrowth. When we got to the road we were stopped at a checkpoint but we told them we had come from Parachinar, and they had to believe us because we speak Pushtu like all the people round here.

"We came because the fighting is still continuing and we were afraid about new attacks by America. There is no work in Afghanistan. We may get food to eat one day but we don't know if there will be anything to eat the next. Here I can work and send money back to my family."

How did he feel about the Taliban? "For the past three months we don't know who is ruling us or who is going to rule us. In our country, no government is helping us, neither the Taliban nor anyone else; if they were helping us, why should we leave? Because of years of drought our lands are useless." Twenty-five miles down the road we stopped in the bazaar of a town called Hangu for tea in a gloomy tea shop full of bearded Pathans. Any new arrivals in these parts? we asked the huge boss. He brought one of his waiters to the table, a youth in his late teens. We learned that he and nine members of his family had come 10 days before, "when the border was not closed". "Many have come, they are staying wherever they can, wherever they have family or friends," he said. "They have not gone to the refugee camps because they are afraid the authorities would push them back." He came from Logar district, from a village 25 miles east of Kabul. "After the attacks in America, the rumour spread that the Americans were going to attack so we came here as fast as we could. We are staying with relatives here. Ten days ago it was easy to slip across the border – now the security is too tight, people can't come through.

"Ten to fifteen thousand people came over in the days after 11 September, but they are all in hiding. Yesterday our president [Mullah Mohammed Omar, the Taliban's supreme leader] ordered people not to leave the country. But conditions are already so bad: the long drought, and winter is coming. We don't have money for winter clothing. If fighting breaks out again we didn't know how we would manage."

I went into a little public call office next door and got talking to the young men waiting to phone. "This guy has just come over," one of them said, referring to a heavily bearded young Pathan in their midst. He, too, was from Logar district. Why had he come? "There is no work in Afghanistan, bread is very costly." Did he plan to stay? He nodded emphatically. "My wife is happy to be here because of the fighting on the Afghan side. I want peace. And I want a job. Give me a job, will you?"

The new refugees I encountered with so little effort were the lucky few. The ones I did not meet, the great bulk of the iceberg below the water, were those thrown out or turned back in recent days by Pakistan, those on the move from remote parts of Afghanistan towards the dim hope of food and safety, and the tens of thousands trapped on the wrong side of the Pakistan border when Pakistan decided it would take no more. All now face the onset of winter cold without money, food, shelter, fuel or adequate clothing. One month ago, the United Nations High Commissioner for Refugees (UNHCR) believed there were nearly one million internally displaced persons (IDPs) in Afghanistan; now they believe there may be up to seven million, roughly half the population.

As long as Pakistan keeps its border closed, the tragedy befalling those millions will remain unseen; and unless the nations of the world pile pressure on Pakistan to reopen the border, and inundate the UNHCR with funds – the agency has appealed for $252m (£176m) to tackle the crisis – the deaths of many hundreds of thousands will occur well away from reporters and television cameras. But it is a tragedy that will come back to haunt us.