The books are very important. Stolen Throne is absolutely essential to understand what Loghain's character was really about, and even ultimately why Cailan and Alistair turned out the way they did, through what you learn about Maric, Rowan and the Guerins. The Calling is essential to understand who Duncan really was beyond the guy who recruited you and then died, and why protecting both Cailan and Alistair was so important to him. Asunder resolves hanging threads from Origins and introduces characters who will definitely show up in Inquisition, and along with Masked Empire set the events that directly lead to DA:I.
Plus, they're all very entertaining reads, well written, and the universe is cohesive between each book and game, really expanding the universe with rich backstories rather than muddling it up. The mess that were those ME books doesn't even compare.
The comics are a different story. They're good, but they follow a unique canon that doesn't blend in so neutrally as the book universe does with the games, and some decisions in the treatment of certain characters are... questionable. Worth reading, especially if you're interested in how Alistair fares post Origin, but take them with a grain of salt and some suspension of disbelief if their canon doesn't exactly match up with yours.
But yes, the books. Read them. Do it. If you're a slow reader or only have limited time or resources to get all the expanded universe, I'd really recommend letting everything else for later and just reading Asunder and Masked Empire now, if you can. They're not mandatory, but they really help situate you on where the world stands right now -it's changed a lot since the last time you saw it at the end of DA2- and no summaries will do the characters and events of these books any justice.