Abramson's departure was announced by the US daily's publisher Arthur Sulzberger, and the paper's own report said: "The reasons for the switch were not immediately clear."
The sudden departure left many questions unanswered both inside and outside one of the nation's most prestigious news organizations.
The New Yorker cited unnamed sources as saying that Abramson quit after a confrontation over her pay, said to be lower than Bill Keller, her predecessor as executive editor and previously managing editor.
"She confronted top brass," the magazine cited a close associate as saying, adding that this may have fed into the management's narrative that Abramson was "pushy."
The report was backed by NPR media correspondent David Folkenflik, who said he had confirmed Abramson "did indeed challenge corporate brass over what she saw as unequal pay" and that Times staff wondered if gender had a role in her ouster.
But the Times disputed that account.
"From the start, Jill's compensation as executive editor was directly comparable to Bill Keller's compensation as executive editor," spokeswoman Eileen Murphy told AFP.
"Today's change in leadership was a result of Arthur Sulzberger's concern about aspects of newsroom management."
Northeastern University journalism professor Dan Kennedy said the departure appeared irregular.
"This was definitely not a normal change," Kennedy told AFP.
"Abramson did not address the staff and was all but hustled out of the building. The explanation that Arthur Sulzberger offered about newsroom management seems completely inadequate."
Dan Gillmor, a journalism faculty member at the Arizona State University, also expressed skepticism.
"If top exec in any other important industry left under mysterious circumstances, @NYTimes would launch a team of reporters to find out why," he tweeted.
Alan Mutter, a former newspaper editor who is now a consultant, said of the abrupt change: "We don't know if she jumped or was pushed but the meager information available suggests the latter."
Mutter added that the ouster "suggests a dissatisfaction with either her style and/or performance as a manager -- and probably does not reflect a reaction to the ongoing secular challenges facing newspapers in general and the Times in particular."
Blazing a trail
Abramson was appointed to head the 160-year-old paper in 2011, and led it in a period during which it was seen as having weathered the transition to digital better than many competitors.
"We successfully blazed trails on the digital frontier and we have come so far in inventing new forms of storytelling," she said in a statement from the paper confirming her replacement.
"Our masthead became half female for the first time and so many great women hold important newsroom positions."
Before taking the top job, the now 60-year-old journalist had been an investigative reporter for the rival Wall Street Journal and then the head of the Times' Washington bureau from 1997.
She acknowledged in an interview last month that she had four tattoos including a "T" representing the Times.
Her replacement, Baquet, is a 57-year-old newspaper veteran and former editor of the Los Angeles Times.
Last year, The New York Times boasted the largest daily and Sunday circulation of any seven-day newspaper in the United States, with a weekday circulation of 1,926,800 print and online versions.
According to the company's 2013 annual statement, the firm had an annual turnover of $1.57 billion.
But like many dailies, the "gray lady" of US journalism has struggled with the move away from print.
The Times has been hit by declining print sales and advertising, and said last year it takes in more revenue from readers than from advertising, in a major shift.
The company also sold off The Boston Globe and other regional newspapers to focus on its core operations, and also divested other assets, including its stake in an online employment website.
The company brought in former BBC chief Mark Thompson, who became president and chief executive at the Times in 2012, as part of its effort to manage a digital transition.
In a memo to staff, Sulzberger said the change comes "at a time when the newsroom is about to embark on a significant effort to transition more fully to a digital-first reality."