Opener Brendon McCullum was unbeaten on 38 at the break with Daniel Flynn (21 not out) after New Zealand elected to bat at the Galle International Stadium. The pair have so far added 40 for the unfinished fourth-wicket stand.
Guptill was caught by Angelo Mathews who took a low catch to his right at second slip, while Williamson was caught at first slip by Tharanga Paranavitana in the same over.
Fast bowler Nuwan Kulasekara put more pressure on New Zealand when he bowled skipper Ross Taylor (nine) in his fifth over to send the tourists reeling at 40-3.
McCullum played some attractive shots in the morning session, hitting Eranga for two successive fours early in his innings. He has so far cracked six boundaries in his 80-ball knock.
New Zealand went into the match with three pacemen, Trent Boult, Doug Bracewell and Tim Southee, and one specialist spinner, Jeetan Patel.
The hosts named two fast bowlers, Kulasekara and Eranga, and as many spinners, Suraj Randiv and Rangana Herath, in their team.
Sri Lankan opener Dimuth Karunaratne, who replaced injured Tillakaratne Dilshan, made his Test debut.