Can you be intolerant to refined wheat and not wheat ?

Anything is possible. Sometimes the refined wheat can have other things added to it in the refining process, and you could be intolerant of those additives.