Actualy I think we share the same idea but we use different words for the same thing. I agree with everything you just said if story is replaced with lore and plot replaced with story. Imho, there's no difference betwen plot and story and I guess you make little to no distinction between story and lore. this is just play word. (maybe this is because I don't usualy speak/write english so perhaps I don't understand some shades)
Oh, I make a further distinction between story and lore. LotSB would be lore, because it's not related to the main story and is instead focused on the workings of something within the universe but not within the plot, and although it does concern the past/ambitions one of the main characters, I'd argue that's still too tangential a connection to warrant as being part of what I'd call the story itself.
So, using my LotR analogy, plot would be "destroy the ring", story would be "Sauron is evil and he wants to conquer Middle Earth using the Ring, which corrupts all who use it with promises of power", and lore would be knowing that Gandalf once saved Gwaihir, lord of the Eagles, from a poisoned arrow, and that's why Gwaihir saves Gandalf at Isengard. If that makes sense.