I don't know....it may just me being a worry wort. This has happened to me before....and I just don't want it to happen again.
IT kind of sounds like he's saying he's not sure he wants to wait anymore, I think you need to talk this over with him instead of leaving it open to interpretation. Ask him if he seriously wants to go back on that decision, but make it clear you don't want to and are not ready to have sex even if he has changed his mind.
Do not give into him. If he's worth it, he'll wait until you're ready, or for marriage if thats when you really want to wait until.
If he's not prepared to, I'd say get out.