2y ago

Is it really a breaking change if a method changes output after an update?

This might seem obviously "yes" at first, but consider a method like foo.debugRepr() which outputs the string FOO and has documentation which says it is meant only to be used for logging / debugging. Then you make a new release of your library and want to update the debug representation to be **FOO**.

Based on the semantics of debugRepr() I would argue that this is NOT a breaking change even though it is returning a different value, because it should only affect logging. However, if someone relies on this and uses it the wrong way, it will break their code.

What do you think? Is this a breaking change or not?

23 comments

This is https://www.hyrumslaw.com/.
Basically there are two types of breaking changes:
The change may break something.
The change breaks a contract of the code.
What you are experiencing with debugRepr() is that you have triggered 1. You have made a chance that may break a user. But you have not triggered 2 because the new output is still within the previous contract. What level of stability you want to uphold is up to you.
- Which one is important is going to depend on the context for sure.
  If it's an open source library, they probably won't care about 1.
  If you're working on internal software used by other developers within the company, management probably really does care about 1 because it's going to impact their timelines.
  If you're working on a proprietary user-facing API, then even if it doesn't cost your company anything management might still care because it could piss off valuable customers.
  I think that, for what ever decision OP is trying to make, looking at that context is more important than quibbling over what exactly constitutes a "breaking change."

https://xkcd.com/1172/
People depend on your program behavior. If you change how the interface works you're going to break people's programs. No way around it. Even if you warn them

You're clearly stating this is debug code, so the user should read that as Here be dragons. Otherwise you may break your user's code, e.g., sysadmins that rely on the logging for monitoring.

has documentation which says it is meant only to be used for logging / debugging
No, it's not a breaking change IMO. The method contract (the "debug" name, the comment) heavily implies the output may change and should not be relied upon.

This one is a bit tricky, because you have to think about logging as an output or a side-effect. And as an industry, we’ve been learning that we should limit the amount of side-effects that our code generates.
If logging is getting ingested by downstream systems like CloudWatch, or other structured logging systems, it is potentially going to be used to detect service issues and track overall service health. These are logs that are serving a functional purpose that is not purely a side-effect, or for debugging forensics.
If this is the case, then you should have a unit test asserting that a log entry is emitted when a method is called. If writing that test is a low or non-priority, then even if it’s a “breaking change,” then that’s a sign that it’s not actually going to break anyone.
I’m sure there’s some monadic view of how to package up the “side-effect” logging as part of a function’s output, but it’s probably annoying to implement in most languages.

Only if it's specified and documented as part of a contract with the user. If they're relying on internal implementation details, well that's a good lesson for them not too do that.

Edge case: If you call foo.version() and it returns a different string in version 1.02 than in version 1.01, that is not a defect.
- That's not a functionality change. The method still does the same thing: "outputs the current version of the software".
  
  I think that's what they're saying.
- Very interesting. Good point!

As a practical matter it is likely to break somebody's unit tests.
If there's an alternative approach that you want people to use in their unit tests, go ahead and break it. If there isn't, but you're only doing such breakage rarely and it's reasonable for their unit tests to be updated in a way that works with both versions of your library, do it cautiously. Otherwise, only do it if you own the universe and you hate future debuggers.

It's your project, do whatever you want.
If changing any observable behavior meant a breaking change, then you couldn't ever change anything. Even a bug fix changes observable behavior. Some people don't seem to be considering that here...

If I don't break the contract and your code breaks that's your problem imo

"if someone relies on this and uses it the wrong way"
The wrong way for you might be the perfect solution for someone else. Once things are being used, you have no idea how people will use them, and they will likely use them in ways you didn't anticipate.
- If you go using code in a way contrary to its documentation, you can't expect semantic versioning to have semantic value to you.
  Nothing in there stops you. You are perfectly free to hack anything in your code. But it's completely outside of your relationship with the author, and frees him from any problem an update may cause you.

For debug code, anything goes. For API, it should be versioned if you're worried about this kind of thing. Such as /v2/ in the path, or a version property on a returned object.

IMO it doesn’t really matter what you said the method was for. If you change the format of a string that is returned by a method that returns a string, there’s a risk of breaking user code, even if it’s just in the context of their dev environment.
Philosophically, whether or not the behavior of your API has changed is completely disconnected from whether or not others are using it “right”. If I can depend on a function to return a certain type of value when given certain arguments, and if it doesn’t produce other side effects, then it doesn’t matter what the docs say or what the function is named, I can use it in any context where I need that type of return value and have this type of arguments available. This type of function is just mapping data to other data. If you modify the function in such a way that the return value changes after being given the same arguments, that’s a breaking change in my book.
- What about version()? Every minor / patch update would be a breaking change.
  
  The way I do it, patches are backward-compatible bug fixes. Minor versions are additional features that don’t change existing functionality. Major versions include breaking changes. I totally get that it seems crazy to bump to another major version just over a string format change. But overall the philosophy works well IMO.

Breaking change. It’s gone from plain text to a markdown formatted text (possibly). There’s changing an interface (obviously a breaking change) and then there’s changing the semantics of a function. I just dealt with a breaking change where a string error value changed for an account registration api call. Previously it returned EMAIL_IN_USE and now it returns EMAIL_TAKEN. Same data type but it broke the client code. Changing values or formats is a breaking change. In your case the documentation says don’t rely on this function for anything but once the output is in the wild any monkey can start using it for anything and it can’t be certain that some code documentation will be consulted before deciding to depend on it.

My tests that observe output from the method are failing so it's a breaking change. Did you not test the printed output?

23 comments