Prayers man.
We brought in a dept of data scientists when we bought a company.
I never witnessed the worse code ever.
I don’t know if they were all 1y bootcamp grads or something. I thought math formulas was supposed to be elegant. But everything was written top-down, no reusability, like coding excel formulas that feed one cell into another and takes a hour or two to calculate. I nope’d so fast out of one “system” they built which was just daisychaining SQL queries into a new file that is fed into another SQL query repeat 7 times.
Mathematicians and scientists are notoriously awful programmers. They get shit done but with absolutely 0 regard to good practices and reusability.
It’s kind of like hiring a bunch of physicists to design a bridge. It’ll probably be a great bridge, but also probably overly complex, expensive and late. And for some reason made out of a novel alloy.
I’ve experienced similar.
One thing that helped: separating research code from production.
Research is to answer a decision problem and much of the optimization problems that are in software engineering do not, and should not, apply.
Once the research problem is answered. Reproducing that answer with production quality systems should be it’s own project. This also serves as a reproduction of the science in the research. Satisfying that hallmark of the scientific method.
Course, getting a company to agree to such an arrangement is near impossible. Especially if they have never been crippled by the mismatch expectations of putting research code in production.
As that is an organizational problem not just an engineering problem, good luck convincing management.
An alternative, if I can’t get such an arrangement is the building a platform that supports integration of research code. That can be… uh… hard. Aside from the people challenges not all tech can support such a platform.
