a) If Masha asks all three different questions, then for the numbers a, b and c thought by Vitya, she will know the coefficients of the polynomial
p(x)=(x−a)(x−b)(x−c)=x3−(a+b+c)x2+(ab+bc+ac)x−abc.
Solving the cubic equation p(x)=0 (for example, using Cardano's formulas) she can find the numbers a, b and c.
b) Suppose Masha has a strategy that allows her to find out the numbers in no more than two questions. Consider three options for the game process, depending on Masha's first question.
1) If Masha first asks the product of numbers, let Vitya answer "0". If further Masha wants to know the sum of the numbers then after the answer "0" she cannot distinguish triples (0,1,−1) and (0,2,−2). And for the sum of pairwise products after the answer "12" it's impossible to distinguish triples (0,2,6) and (0,3,4).
2) If Masha first asks the sum of the numbers, let Vitya answer "0". If further Masha wants to know the product of numbers, then after the answer "0" she cannot distinguish triples (0,1,−1) and (0,2,−2). And for the sum of pairwise products after the answer "-49", it's impossible to distinguish triples (0,7,−7) and (3,5,−8).
3) If Masha first asks the sum of pairwise products of numbers, let Vitya answer "-18". If further Masha wants to know the sum of the numbers, then after the answer "3" she cannot distinguish triples (0,−3,6) and (2,5,−4). And for the product after the answer "-72" it's impossible to distinguish triples (3,−4,6) and (2,−3,12).
In each of the options, after two questions, Masha doesn't have enough information to determine three numbers unambiguously, therefore she does not have a strategy that would allow her to determine the Vitya's numbers in two moves.