Theresa May will meet Jeremy Corbyn later to see whether there is common ground to break the Brexit deadlock.
Brexit Secretary Stephen Barclay said the “remorseless logic” of MP numbers in the House of Commons meant the UK was heading for an “undesirable” soft Brexit, or closer links with the EU.
Mr Corbyn says he wants a customs union and workers’ rights to be priorities.
The PM’s move to hold talks has angered some Brexiteers, with Wales Minister Nigel Adams resigning his role.
In his resignation letter, Mr Adams said the government was at risk of failing to deliver “the Brexit people voted for”.
The PM is also due to meet Scotland’s First Minister Nicola Sturgeon, who supports the option of a public vote.
Following the PM’s announcement, prominent Brexiteer Boris Johnson accused Mrs May of “entrusting the final handling of Brexit to Labour”.
And Jacob Rees-Mogg, leader of the Eurosceptic European Research Group (ERG) of Tory MPs, described the offer as “deeply unsatisfactory”.
But Mr Barclay blamed hard Brexiteers in the ERG who refused to back the PM’s deal.
He told BBC Radio 4’s Today programme the alternative to the PM’s deal was to seek votes from the opposition benches “because 35 of my own colleagues would not support the prime minister’s deal”.
He said the consequence of MPs not passing the PM’s deal was either a “soft Brexit or no Brexit at all”, saying: “It [a soft Brexit] is undesirable but it’s the remorseless logic of the numbers of the House of Commons.”
Mr Barclay said the EU has said the withdrawal agreement is the only deal available, but he said Labour had expressed more concern about the future relationship – which is contained within the separate political declaration.
The withdrawal agreement includes how much money the UK must pay to the EU as a settlement, details of the transition period, and citizens’ rights – as well as the controversial Irish backstop that aims to avoid a hard border on the island of Ireland.
The political declaration sets out proposals for how the UK’s long term future relationship with the EU will work after Brexit.
The BBC’s Laura Kuenssberg said there was not much difference between the government’s version of Brexit and Labour’s version – but there did not seem to be “an enormous amount of confidence” a political consensus could be reached from either party.
Mr Corbyn said he was “very happy” to meet Mrs May and recognised his own “responsibility” to try to break the deadlock.
But the meeting is not expected to take place before this afternoon, at the earliest, says our political editor, who was told by Mr Corbyn’s team that he was not available on Wednesday morning for talks with the PM.
It comes after Mrs May had more than seven hours of talks with her cabinet on Tuesday.
Laura Kuenssberg said there was “rage and dispute” in the cabinet meeting, with “wildly varying accounts” of how many people were for and against different versions of Brexit extensions.
Labour has previously said it has five tests for judging any final Brexit deal, including protecting workers’ rights, establishing a permanent customs union with the EU and securing the same benefits of being in the single market the UK has currently.
- Labour’s five demands for supporting a Brexit deal
Shadow business secretary Rebecca Long-Bailey told the Today programme that Labour was approaching the conversations with “an open mind”.
Later in Parliament
Meanwhile, a cross-party group of MPs will attempt to push through legislation to stop a no-deal Brexit.
If passed into law, the bill – presented by Labour MP Yvette Cooper – would require the PM to ask for an extension of Article 50.