Schengen: what can they ask, and what am I obliged to tell?

Bleary eyed and grumpy last week due to a delayed night train, I was nevertheless rather surprised when a Danish policeman demanded to see my passport abord the night train to København H at Padborg (border station from Germany). When I asked him why and how he was demanding this his answer was "I can". My response was to question why this was so, as both Germany and Denmark are in Schengen. "It's the same if you take flights within the EU" he said to me. "No it's not" was my response, "that's to prove to the airline that the right person is travelling".
Anyway, not wanting to antagonise the others in the compartment, I showed the passport and the policeman pottered along to the next compartment, checking every passport systematically.
But - as I understand Schengen - this is not right.
Firstly, border controls are allowed to be random, but they are not allowed to be systematic, and this most definitely was systematic.
Second there is the question of what documentation I would actually be required to show. Because if this were just a regular police control that happened to be at the border, then I would only have to prove my identity according to whatever standards were applicable in the state I was in (see the Brenner case I documented a few years ago). So in Denmark that could be a sundhetskort with a CPR number on it, and would not need to be a passport.
But there is a dreadful lack of clarity here. There's basically nothing on the Commission's website at what can and cannot happen at internal borders. Wikipedia has little more. The best I can find is Article 21 of the Schengen Borders Code (PDF here)
The abolition of border control at internal borders shall not affect:
(a) the exercise of police powers by the competent authorities of the Member States under national law, insofar as the exercise of those powers does not have an effect equivalent to border checks; that shall also apply in border areas. Within the meaning of the first sentence, the exercise of police powers may not, in particular, be considered equivalent to the exercise of border checks when the police measures:
(i) do not have border control as an objective,
(ii) are based on general police information and experience regarding possible threats to public security and aim, in particular, to combat cross-border crime,
(iii) are devised and executed in a manner clearly distinct from systematic checks on persons at the external borders,
(iv) are carried out on the basis of spot-checks;
(b) security checks on persons carried out at ports and airports by the competent authorities under the law of each Member State, by port or airport officials or carriers, provided that such checks are also carried out on persons travelling within a Member State;
(c) the possibility for a Member State to provide by law for an obligation to hold or carry papers and documents;
(d) the obligation on third-country nationals to report their presence on the territory of any Member State pursuant to the provisions of Article 22 of the Schengen Convention.
Now the checks at Padborg clearly contravene (a) and are not, in Denmark, covered by (c). But (c) concerns countries where there is an obligation to show an identity document when demanded to do so by police.
Anyway, via networks of bloggers I am going to try to assemble a Schengen internal borders guide - essentially what can be demanded of anyone, anywhere when crossing an internal border in the Schengen zone. Please do contribute your experiences in the comments.
In the meantime I'm putting a copy of Article 21 in Danish, German, French and English in my passport wallet!