If I'm making smalltalk with my SO, it's because there's something more weighty I want to discuss, but I'm looking for a way to broach that topic in a better way. So regardless of how I feel about them, it's still an issue because it means I don't feel safe to attack the topic head-on.
That feels like a communication strategy tailored to your relationship rather than small talk. small talk, in my opinion, is meaningless banter without motive. This may be pedantic, but I just don't think I would classify that as smalltalk, since there's an objective and I'm theoretically engaged, I personally would have no issue with it.
It's when there's seemingly "no point" that I consider it difficult