The only times that I know Cullen isn't in his office, for certain, are:
1. When you first arrive in Skyhold Cullen is in the lower courtyard, where the tents are set up for the wounded refugees and soldiers.
2. After a few conversations with him you'll find a Guard in his office that tells you he went to speak with Cassandra. You find the two of them speaking in the building where all the blacksmiths are (it is to the right of where Cassandra normally is hanging out).
While in Skyhold, if you check your Quest Map where it shows you all of the NPC icons, do you see Cullen anywhere on there? On one of the 3 levels?
Might be a bug, don't know. Without knowing all the stuff you have or have not done in the game and actually being able to mess around with it, it's hard to troubleshoot.
In case you are checking the wrong place for his office, if you head up the stairs onto the ramparts to the left of the inn/tavern, and then go left when you get to the top of those stairs, through 2 doors and you should be heading directly toward his office at that point...
Sorry I cannot be of more help. 