HM Revenue and Customs must avoid the impression it had "one rule for the rich and another for everyone else", the Public Accounts Committee said.
It called on the government to ensure the rules over footballers' image rights were tightened.
An HMRC spokesman said the rich got extra scrutiny, not special treatment.
The report highlighted particular concern about tax evasion in the football industry and the "misuse" of image rights to reduce tax liabilities.
The committee revealed that 43 players, 12 clubs and eight agents were currently the subject of "open inquiries" by HMRC. However, it said that in some cases clubs were failing to co-operate fully with the tax inspectors.
"We were appalled to hear that not all football clubs are providing HMRC with data under a voluntary agreement struck with the English Premier League," said the report.
Current tax rules allow for income from image rights to be treated as a separate revenue stream for tax purposes.
It means that taxpayers who believed their image had a market value could set up a company to receive payments for those rights.
The chief executive of HMRC, Jon Thompson, told the committee in December that he would like to see a review of the way footballers can reduce their tax bills. He said payments to them for their image rights were "the most significant risk in football" faced by HMRC.
The committee feared these rules are being abused. "Government should take urgent action to address image rights taxation," the report said.
Since HMRC set up a specialist unit dealing with so-called high net worth individuals - from across all walks of life - in 2009, the amount of income tax they paid had fallen by £1bn, the report said. This was despite income tax receipts from the public as a whole rising by £23bn over the same period.
Since 2009, each of the estimated 6,500 individuals worth £20m or more has been assigned a "customer relationship manager" by HMRC to administer their tax affairs.
But while HMRC said this had resulted in the collection of an additional £2bn in tax. it was unable to explain why the income tax they paid fell by 20% - from £4.5bn in 2009-10 to £3.5bn in 2014-15 - when the overall income tax take rose by 9% to £23bn.
The PAC, which took evidence from Mr Thompson and other experts, said it was "alarming" that at any one time about a third of high net worth individuals were likely to be under inquiry for unpaid tax.
But the HMRC had a "dismal record" when it came to prosecuting the very wealthy for tax fraud in the criminal courts, the committee said.
In the five years to 31 March 2016, it completed just 72 fraud investigations into such individuals, with all but two having been dealt with using its civil powers. Only one case resulted in a successful criminal prosecution.
Of the 850 penalties issued to the very wealthy since 2012, the average charge was £10,500, a figure the MPs said was unlikely to be a deterrent to multi-millionaires.
PAC chairwoman Meg Hillier said: "If the public are to have faith in the tax system then it must be seen to have fairness at its heart. It also needs to work properly. In our view, HMRC is failing on both counts.
"HMRC's claims about the success of its strategy to deal with the very wealthy just don't stack up."
To re-build trust, HMRC must be far more transparent about its operations and dealings with the super-rich, said the PAC. And it should consider what further powers it needs. The MPs said HMRC must report back to the PAC by July this year.
An HMRC spokesman strongly defended the department.
He said there was "absolutely no special treatment" for the wealthy. "In fact we give them additional scrutiny, with one-to-one marking by HMRC's specialist tax collectors, to ensure that they pay everything they owe, just like the rest of us do," he said.
He added that the department carefully scrutinised arrangements between football clubs and their employees to ensure the right tax was paid.
"In recent years we have identified more than £158m additional tax yield from clubs, players and agents," he said.
0 comments: