[DISCUSS] IBM using LLMs to convert COBOL to Java
[DISCUSS] IBM using LLMs to convert COBOL to Java
IBM's new product offering, Code Assistant for IBM Z, leverages a generative AI model to translate COBOL code to Java.
It's not the 1st time a language/tool will be lost to the annals of the job market, eg VB6 or FoxPro. Though previously all such cases used to happen gradually, giving most people enough time to adapt to the changes.
I wonder what's it going to be like this time now that the machine, w/ the help of humans of course, can accomplish an otherwise multi-month risky corporate project much faster? What happens to all those COBOL developer jobs?
Pray share your thoughts, esp if you're a COBOL professional and have more context around the implication of this announcement 🙏
You're viewing a single thread.
Something I found is that LLM struggle with weirder cases, when it comes to code.
I once tried getting ChatGPT (though admittedly only 3.5) to
generate code inunderstand SaHuTOrEPoL, which is one of the more esoteric languages I created, and it really struggled with it.I don't think the LLM is gonna do that great of a job with it for this reason, but still worth giving a shot. ChatGPT is a well trained coding chimp. You realistically could get a well trained chimp to start off a lot of projects and have people finish it. The fact that it can correct itself after you explain how it's wrong is very powerful as well.
LLM isn't gonna be useful for converting a single program from COBOL to Java, it is gonna be useful for converting many programs from COBOL to Java. I bet IBM is trying this on their own shit first before they try to sell it to customers, because language conversion software would be a HUGE but very boom based money.
As I stated on a different comment in this thread, I worded my comment poorly. Why I think this is relevant however that, at least in this case, if an LLM get code which is significantly different from what its trained with, it can make wildly incorrect guesses. While here its because of a language with a... unique syntax, I think this could also be the case for code with a lot of technical debt or weird design decisions.
Yeah, I don't think this will be as good as we want. The amount of technical debt I'd expect in these big programs would be off the charts. It would take a full team of people just to feed in corrections, and they would need to both know COBOL and be willing to kill all the COBOL jobs to do it. It's a tough ask
I'm still optimistic though because it's IBM and converting between languages is a billion dollar question. I don't think IBM would do a truly irresponsible deployment of such big corporate changes. I'd expect IBM to do a LOT of testing before sending anything out that would actually change the world, such as banking software. But in could be wrong.
IBM hawks new conversion tools all the time. None of them are amazing sliver bullets, all of them require humans to comb over the resulting output. And every single one I’ve ever used chokes on any weird case.
From the RPG fixed form to free form, DDS to DDL conversion, and so on all of them are usually more trouble to use than to not use.
IBM does this kind of stuff all the time. And for some folks it’ll work some of the times. But at this point, I just skip any WS tool they put out and have a snippet on RDi and RDz that does all the required plugging away to call web services from the COBOL module.
Probably because it wasn't trained on it. That's not any different from asking the same of a human.
Sorry, I worded my comment poorly, see my reply to FaceDeer in this thread
Why would you expect ChatGPT to know how to write code in a language that you created yourself? According to that repository you created it last year, and ChatGPT was only trained on data up to 2021 so there's no way it could have been in its training set. Even though AIs have surprised us with their insights in a lot of cases they aren't magical.
Admittedly, I worded my comment poorly. What I meant is that ChatGPT struggled with understanding the semantics and structure of the language.
As an example, try from this this code block
$S__ do S__-m__w("Hello world!") doYou can, hopefully guess that
S__is a variable which has a methodm__w, accessed by using a hyphen, rather than a dot and statements end using adokeyword. ChatGPT missed on all marks.Have you tried including the docs in the prompt as well? I wouldn't have guessed that
S__is a variable in your example, but I would've known that after reading the docs. I'd expect ChatGPT to at least do OK if it had access to the docs.Huh, fair enough, its hard to know whats obvious and whats not.
As for the second part, to my slight surprise, it did manage to figure it out. I guess I just suck at using LLMs lol
edit: To clarify, previously I tried only to correct its mistakes over and over again untill I got frustated
edit2: I still feel like chatGPT is struggling with basics of the language. maybe its just me being shit at using LLMs but smh