Skip Navigation

Karl Marx correctly identified the root problem of production (and invented the concept of tech-debt) and the world has hated him ever since.

First two paragraphs of 18th of The Eighteenth Brumaire of Louis Bonaparte is one of the most if not the most insightful works of sociology.

Hegel remarks somewhere[*] that all great world-historic facts and personages appear, so to speak, twice. He forgot to add: the first time as tragedy, the second time as farce. Caussidière for Danton, Louis Blanc for Robespierre, the Montagne of 1848 to 1851[66] for the Montagne of 1793 to 1795, the nephew for the uncle. And the same caricature occurs in the circumstances of the second edition of the Eighteenth Brumaire.

Men make their own history, but they do not make it as they please; they do not make it under self-selected circumstances, but under circumstances existing already, given and transmitted from the past. The tradition of all dead generations weighs like a nightmare on the brains of the living. And just as they seem to be occupied with revolutionizing themselves and things, creating something that did not exist before, precisely in such epochs of revolutionary crisis they anxiously conjure up the spirits of the past to their service, borrowing from them names, battle slogans, and costumes in order to present this new scene in world history in time-honored disguise and borrowed language. Thus Luther put on the mask of the Apostle Paul, the Revolution of 1789-1814 draped itself alternately in the guise of the Roman Republic and the Roman Empire, and the Revolution of 1848 knew nothing better to do than to parody, now 1789, now the revolutionary tradition of 1793-95. In like manner, the beginner who has learned a new language always translates it back into his mother tongue, but he assimilates the spirit of the new language and expresses himself freely in it only when he moves in it without recalling the old and when he forgets his native tongue.

Here Karl Marx also humanistically describes the root problem of production of processes in general. The problem of the cumulative error. While Marx is describing the production of history that is a productive force, it is also in a class of meta-productive forces. These are productive forces that do not directly produce material goods but affect the productive forces that do create physical goods and services.

This is literally the same metaphor as tech-debt. Software has many parallels because the production of software is also a meta-productive force. It produces an immaterial good that is productive force in itself and produces other immaterial and material goods. Tech-debt is the effect of inefficient cycles, that weighs on future cycles.

This is similar to the educational concept of Wittigenstien's ladder or lying to children, because learning itself is a meta productive force. In essence as a productive force, learning is cyclical. In order to understand how atoms function, we teach children models that are technically incorrect such as the Bohr model so they can understand approximations closer and closer to the truth. These are all productive cycles that compound on each-other.

In all these cycles the most important thing is that each future cycle must trend more strongly in the positive direction (a better understanding, better materialist outcomes, software that is cheaper to build maintain and is more resilient), rather than accumulating errors, which will eventually self-reinforce to middling and negative results and eventually collapse.

These cycles represent real risk in the real world. For example one of the most difficult things about the way we develop technology is that its not deterministic. The classic black swan scenario of solar flares causing a strong EMP would result in chaos. This is not just because would have to rebuild everything. The simple reason fact is we cannot "rebuild" the world ex nihilio because we don't have clear knowledge of certain steps along the way. Sure we could figure it out, but that's a different process. That's the process of technological discovery, not the process of applying technology.

You can think of it as rebuilding a neighborhood being hit by a bomb. The neighborhood is gone and we don't have floor plans of some of the key buildings. But it's actually worse than that. We don't have plans to the buildings that lead to the creation of those buildings. We don't have a university, but to build a university we need a foundry, we need a printing press, we need a quarry, we need a brick yard, etc. We don't have some of those, or some of the inputs to some of those. It's actually more complex than that because each of these "functions" also scales and has dependencies at different levels of scale. So we might be able to build a brick yard, but that brick yard won't be able to produce bricks that meet modern specifications until we build a smaller university to research brick making.

I literally think about these daily during the course of my software job:

  • the first time as tragedy, the second time as farce
  • Men make their own history, but they do not make it as they please
  • The tradition of all dead generations weighs like a nightmare on the brains of the living.

There was a period of time where I played nothing but Dark Souls 1 and 3 (same theme) and thought about these.

25 comments
  • Good post. I had an adjacent thought just like this the other day, while I watched the progress bar of a cargo build command. I watched it build 700+ other crates so that this one project could be built. I thought, "How the fuck does anything actually get done, this is simply insane. How many people does it actually take to write this software...". Naturally, something wasn't compiling correctly (thanks windows) and I shelved the thought for now.

    It makes me think of all the MRI machines still being controlled by network isolated Windows XP installs.

    • I thought, “How the fuck does anything actually get done, this is simply insane. How many people does it actually take to write this software…”. Naturally, something wasn’t compiling correctly (thanks windows) and I shelved the thought for now.

      I weep that the tooling space is so bad out there that we're not treating packages as first class objects and working over packages is a crime against technology. NX is approaching this capability, but they're still not really at the point of a solid general maintainable pattern and approach. It should not be this hard in 2025 to manage 25 domains as individual packages, use semantic versioning, commit lint, and automatically bump things. It should not be this hard in 2025 to apply the same tool configuration change to all of them. It should not be this hard in 2025 to ensure they're all uniform based on their "types".

      It makes me think of all the MRI machines still being controlled by network isolated Windows XP installs.

      I used to work in a tangential space. It has only gotten worse. Windows hasn't been supplanted. These companies have simply created more complexity and fragmentation in their own pipelines when they've upgraded to the embedded/kiosk versions of newer Windows OSes.

      During the move to Vista / 7, windows virutalized sound and never made direct bindings. I was working on real time software at the time that created experimental and medical tools for EEGs. The jitter and latency was insane when trying to match up the EEG data to the point in time of the experiment in real time. That software works on linux now but unfortunately there's a hardware gap. A solution on windows is enforcing the use of Realtek cards whose drivers have direct bindings, this has the problem that it's a non-standard per-vendor interface which creates lock-in, bugs and dependency.

      Imagine tomorrow ALSA and JACK go away and there's only PulseAudio/Pipewire, nobody would stand for that shit in the Linux community.

  • I like how reading Marx helps you understand your software job and I love the colorful metaphors. For some friendly nitpicking:

    While Marx is describing the production of history that is a productive force, it is also in a class of meta-productive forces. These are productive forces that do not directly produce material goods but affect the productive forces that do create physical goods and services. This is literally the same metaphor as tech-debt. Software has many parallels because the production of software is also a meta-productive force.

    Is it really necessary to use the term meta-productive force here? I get, that software seems immaterial, but it's still a commodity, just like a table or some wheat or linen. Even software services are commodities that are like any services consumed while being produced. So software workers are workers and part of the productive forces, exactly like engineers or mechanics. So with tech-debt you are thinking about a problem in the realm of production, the base of society.

    What Marx is describing here seems at first to be in the realm of ideology: how revolutionaries talk about themselves and style their movements. So it's more about the superstructure than the base. So it seems to be about a totally different realm and not applicable to tech-debt at all. Though, I get that the similarity is still helpful. And there might be an actual similarity on another level, that explains it:

    Mentioning Hegel, Marx hints at the underlying dialectic at play in history. And dialectical thinking might actually help a theoretically sound analysis of tech-debt. Without going to the trouble of fully doing one, I can sketch it out with Engels three laws:

    • The law of the unity and conflict of opposites: There is a contradiction between short term profit interests leading to the accumulation of tech-debt on the one hand and the need for software as a commodity to have actual use-value on the other hand. This contradiction can not be fully resolved within capitalism. (Edit: it actually goes back the classic contradiction between use-value and exchange-value)
    • The law of the passage of quantitative changes into qualitative changes: Accumulation of quantitative changes (tech-debt) pass into sudden qualitative changes (software becomes unmaintainable profitably and is dropped).
    • The law of the negation of the negation: The negation of proper methods and maintenance sooner or later negates the use of the software and thus this very way of managing it.
    • Is it really necessary to use the term meta-productive force here? I get, that software seems immaterial, but it’s still a commodity, just like a table or some wheat or linen. Even software services are commodities that are like any services consumed while being produced. So software workers are workers and part of the productive forces, exactly like engineers or mechanics. So with tech-debt you are thinking about a problem in the realm of production, the base of society.

      This is largely correct. Even things like knowledge and learning have a commodity form. I was trying to lead people away from "production" = "factory" or thing I use. Production is fully bootstrapped in that production produces itself. Labor is a commodity after all.

      What Marx is describing here seems at first to be in the realm of ideology: how revolutionaries talk about themselves and style their movements. So it’s more about the superstructure than the base. So it seems to be about a totally different realm and not applicable to tech-debt at all. Though, I get that the similarity is still helpful. And there might be an actual similarity on another level, that explains it:

      I get into a clearer example in this comment thread.

      https://lemmy.ml/post/30849595/18912884

      What is written as is is ideological, but it's a generalization of materialism.

      The law of the unity and conflict of opposites: The contradiction between short term profit interests leading to the accumulation of tech-debt and the need for software as a commodity to have actual use-value can not be resolved within capitalism.

      The law of the passage of quantitative changes into qualitative changes: Accumulation of quantitative changes (tech-debt) pass into sudden qualitative changes (software becomes unmaintainable and is dropped).

      The law of the negation of the negation: The negation of proper methods and maintenance sooner or later negates the use of the software and thus this way of managing it.

      Yeah I'm not so great at Engels vocab but I cover each of these in some examples.

      • Not maintaining saw blades on a saw mill
      • Running out of trees
      • TDD as a maladaption that creates tech debt
  • If I can summarize your thoughts, you are basically saying:

    1. The progress of human history necessarily accumulates error
    2. This human-historical error is conceptually similar (or even linked) to the concept of technical debt in software, in the sense that both the structure of society and the organization of production are what you call meta-productive forces
    3. Accumulation of error destabilizes and eventually destroys/obsoletes a meta-productive force
    4. Despite (3), development of productive and meta-productive forces is inevitable, leading inevitably also to accumulation of error
    5. Therefore the development of the productive and meta-productive forces leads constantly to upheaval (something like punctuated equilibrium) which trends overall toward perfection (“truth”) as they are perfected in each cycle
    6. Despite the seemingly inevitable trend toward perfection, there can be events which knock back progress as the “ladder” is kicked after each successive cycle

    Is this an accurate summary?


    Some thoughts:

    The direction of thought is interesting.

    I do not think your excerpt from 18th Brumaire is talking about technical debt, or accumulated error. Marx here refers only to “epochs of revolutionary crisis”. He is not making any claim of historical continuity in general, only in transitional periods. As well, the extent to which the past “weighs itself” on the present is through forms of expression, not necessarily in substance.

    I do not see a point in expanding the concept of productive forces into a new concept of meta-productive forces. Feels like it blurs conceptual lines, at worst maybe confusing the Hegelian dialectic with the Marxian dialectic (production of concepts vs production of material world)

    I understand technical debt not as some ideal accumulation of error, but as a result of the dependence of material productive power with the state of technology at a given moment. I can write more about this later; my transit stop is coming up.

    • Well of course Marx wasn't talking directly about tech debt Marx didn't consider the computer. ;-)

      To expand on it, any process or productive force has inputs, as well as positive and negative outputs. Inputs are things that are needed to complete the cycle like "labor", "material", "knowledge", etc. Positive outputs are things like "knowledge", "goods", etc. Negative outputs are things like "pollution", "waste" (distinct from pollution because some waste is effervescent like wasted labor), etc.

      To simplify this we should actually get rid of "inputs". Inputs are an implementation detail (mainly for amortization and property interests) that can be represented as negative outputs.

      These processes work in cycles, where each previous cycle affects the next cycle. For example I make lumber, I own 5 acres, each year I process an acre of logs (input) and ship 500 board feet(output), after 5 years I have cleared the forest and have no way to produce. Each year the available resources that can act as my inputs dwindles, which is a systemically speaking actually a negative output. So we can reframe this process as each yeah I ship 500 board feet (output) and I cut down an acre (negative output).

      When a cycle begins or ends is not really relevant. These process in reality are continuous. Cycles are merely a form of abstract structure around a long running process that has repetition. A cycle could be a day or a year. It could be until the next time someone jumps into the Kiln of the First Flame. What really matters regarding cycles is the cause between point in time decisions and their effects on the process later on.

      Each cycle can adapt to new conditions, in an optimal/hindsight/longitudinal sense adaptations are changes to the process as a result of a cycle that increase the total amount of possible cycles (leading to longer lived processes), maladaptions are changes that decrease the total amount of possible cycles. For my lumber mill, an adaption (given more acreage) would be planting trees with a 30 year outlook. If I had 31 acres, each year I plant an acre and log an acre, that removes a negative output. A maladaption would be extracting money that would be spent on sharpening saw blades, leading to premature failure of the mill's mechanical systems.

      The overall way we can gauge the health of a process is through relative risk. Risk is a balance of positive outputs, negative outputs, adaptions and maladations. This is a fairly fuzzy, complex and contextual measure. For example if I'm not sharpening my saw blades that increases risk, however how much depends relatively on what the effects are on future cycles, however this risk may likely be a lower risk than the risk of my negative output which has a 5 year horizon.

      These cycles can adapt to output immaterial things like software, social knowledge, individual knowledge, etc.

      Error in this case is metaphorical, error fuzzily represents maladaptations and negative outputs.

      Tech debt is a complex inner process that stems from the production of goals, e.g. what, how and by when to build something. Those goals have implicit effects on the maladaptions and creation of negative outputs during the outer process of actually building the software.

      The tech debt case essentially says building a software has the following outputs:

      • deployments
      • features

      However it has the following negative outputs:

      • labor cost
      • size of code
      • complexity of code

      Tech debt is prioritizing a specific feature output over it's long term negative costs such as increased code size, increased code complexity and increased labor cost to produce more features and lower future labor costs.

      I do not think your excerpt from 18th Brumaire is talking about technical debt, or accumulated error. Marx here refers only to “epochs of revolutionary crisis”. He is not making any claim of historical continuity in general, only in transitional periods. As well, the extent to which the past “weighs itself” on the present is through forms of expression, not necessarily in substance.

      I'd disagree with this mainly because we're getting caught up on the metaphor when its spelled out plainly:

      "Men make their own history, but they do not make it as they please; they do not make it under self-selected circumstances, but under circumstances existing already, given and transmitted from the past."

      This is equivalent to the production cycle of the next set of history is dependent on the outputs, negative outputs, adaptions, and adaptations of the previous one.

      I do not see a point in expanding the concept of productive forces into a new concept of meta-productive forces. Feels like it blurs conceptual lines, at worst maybe confusing the Hegelian dialectic with the Marxian dialectic (production of concepts vs production of material world)

      Historical materialism is already a "meta-productive" force (meta-productive is my flair to get people out of the production = factory mindset. You're right that productive forces already describe what I'm saying, because labor is a commodity thus production is a commodity. It's fully bootstrapped.) through the concept of the base and superstructure. Cyclical changes to the base are based on the previous cycle of material production (what we have), as well as the previous cycle of immaterial production in the superstructure what we want to do with it). Cyclical changes in the superstructure have a mirror of these forces. In essence the whole idea is:

      • that we want is based on what we have and don't have and what we want/feel
      • what we actually produce is based on what we have and don't have and what we prioritize (want/feel)

      These 2 paragraph is are a more general, humanistic and literary version of historical materialism which is more specific.

      I understand technical debt not as some ideal accumulation of error, but as a result of the dependence of material productive power with the state of technology at a given moment. I can write more about this later; my transit stop is coming up.

      The problem here is that you're describing a specific kind of tech debt. I agree that that there is a specific kind of tech debt that exists based on the general availability of tools.

      However the kind of tech debt that typically accumulates in companies is due to adaptions/maladaptions and projected outputs. To offer an analogy, concrete was invented before the invention of the backhoe. That creates a specific kind of tech debt. However regardless of that a company decided to build a house without a foundation because it would be too expensive (negative output) to dig up and move all that earth, and it would prevent the house from being sold before a specific date (avoiding negative outputs). Someone commented above some helpful Engels vocab that hits on this.

      In your example there's also a sub-case based in lack of knowledge. For example choosing test driven development can be an adaption or a maladaption depending on how you actually write the software and the tests. For example if your code is well tested using individual tests and individual fixtures.

      Over the long run that can be a maladaption because that increases the cost to maintain the software. Fixing a specific bug or implementing a specific feature can affect the whole code base and affect all the tests. This is a maladaption because it locks up your process, preventing you from delivering certain features, certain fixes, or improving the process itself. This comes from a lack of knowledge that can be represented as its own negative output or as increased cost/code size/complexity.

      In order to make this an adaption rather than suffer the maladaptive effects the team would also need to adopt practices like factories, shared behaviors, behavior driven development, and discriminating tests based on value. This would allow you to have a highly tested code base, where the test architecture itself allows for sweeping changes. For example lets say we are testing an API that has a set of controllers, each controller gets a context for the API call. We want to change the structure of that context. If we are using fixtures we need to make that change (which can be complex) in hundreds of files. If we are using factories we need to make that change in the factory file, and only the tests that explicitly supply the changed properties of that context.

      So there are ordered effects inside of these processes based on the processes themselves.

      • In 18th Brumaire, when Marx says dead generations weigh on the living, he’s not talking about some accumulation of flaws that have to be contended with. He’s trying to explain why revolutionaries in the 1848 French Revolution chose antiquated modes of expression to describe the revolution itself, which was, of course, a modern rather than an ancient turn of events. It seems totally anachronistic and contradictory; why does this revolutionary movement look both forward and backward?

        The reason is that the past is familiar and the future is not. In that unfamiliar context, it is natural to cling to those old expressions in order to describe what is in substance a brand new thing in world history. Hence the analogy of a new language speaker internally translating everything to their mother tongue for a while, until they are so fluent that they no longer need to do so.

        The line about men not making the world as they please, but on the basis of the past, is necessary to explain this apparent double movement forward and backward in time.

        any process or productive force has inputs, as well as positive and negative outputs […] Error in this case is metaphorical, error fuzzily represents maladaptations and negative outputs.

        A productive force isn’t a process any more than a hoard is capital, or labor power is living labor. Production is the process which sets in motion the productive forces, which can only happen within given relations of production.

        The stuff about positive and negative outputs seems vague and subjective. Same with “inefficiency” which causes “error”. This is heading in an idealistic direction.

        Instead of fussing with these concepts, we should focus on the material forces of production and the relations of production.

        Intrinsic to Marx’s conception of human labor is a distinction between manual labor and mental labor:

        “[W]hat distinguishes the worst architect from the best of bees is this, that the architect raises his structure in imagination before he erects it in reality. At the end of every labour-process, we get a result that already existed in the imagination of the labourer …”

        In parallel with the development of the capitalistic mode of production develops also the division of labor, especially between mental and manual labors. The work of organizing production versus directly producing is thus separated at first privately, internal to a firm. Later, the mental labor (research and development) fully detaches as its own bona fide industry. But this can only manifest if the mental labor can be commodified. Hence intellectual property, consulting, software, etc. A mental product takes on the appearance of a physical commodity in contradiction with its incorporeal reality.

        The point here is that software figures in the productive process as constant capital alongside machinery, supervision/organization, and other fixed technical inputs. Technological progress is nothing other than the relative increase, over time (cycles if you wish), of constant capital to variable capital as a proportion of the product’s value. The capitalist is motivated to do this to receive relative surplus value, by producing more cheaply than the industry average.

        If relative surplus value is received through technological innovation, then technical debt (understood generally) is simply the opposite possibility: technical debt is the accumulation of inefficiencies in terms of value inputs which threaten to make the business uncompetitive. This can be because of code complexity, code size, spaghetti code… or a zillion other industry-specific reasons. But these are mere manifestations of productive inefficiency, and they matter not because of subjective “negative outputs” but because they threaten the one thing that matters to a capitalist: their production of surplus value.

        Put in this way, we don’t need “fuzzy” concepts like error or risk or negative output to understand technical debt.

    • I do not see a point in expanding the concept of productive forces into a new concept of meta-productive forces. Feels like it blurs conceptual lines, at worst maybe confusing the Hegelian dialectic with the Marxian dialectic (production of concepts vs production of material world)

      Yes, that's probably it. I was trying to put my finger on it.

25 comments