Let me rewrite the transtion relation a little bit (but that does not matter):

- Vin={a}
- Vout={o}
- Vstate={p,q}
- I=!!p
- R=a&!o&next(p)&!(q&p&next(q))

So we see that all transitions are labeled with a&!o and all target states must satisfy p. The state transition diagram looks as follows:

Hence, your solution should be fine

init 2,3;
transitions
(0,{a},{},0);
(0,{a},{},1);
(1,{a},{},1);
(1,{a},{},0);
(2,{a},{},0);
(2,{a},{},1);
(3,{a},{},0);

So, I think that your solution is correct and something with the submission or the software is wrong. Please contact your tutor so that you will be given the points that you deserve, and contact the exercise organizer to check what is wrong with your solution. Maybe the software needs to be fixed.