The following is part of a message (anonymized) that ultimately I did not send. It is worth thinking about it though
My implicit assumption when I started was “AI all the things”, with the goal of getting the process and end results of cryptography research served perfectly at the push of a button. My vision was that those results would be fed back to another AI tool to create the end product (either code or robotic if material). That would then spark new research questions that a human needs to abstract and create a research-prompt for. And the cycle continues.
Consequently, to me at least, the “human in the loop” value-add is in the judgement and distillation/synthesis of a worth-while research endevour. The human has to evision what he wants to obtain, think deeply if the goal is a “good” one (both profitable and moral) and then explain it to the machine which will find a way to do that.
I never thought of the consequences of this vision: As labor becomes cheap (AI agents cost much less than humans for now) the “profitable” bar lowers and so in the end we get that a good goal is a moral goal as anything is profitable.
For the record, I do know this is naive and wrong. That this was the basic premise of the scientism of the Enlightment period that created terrible wars and the technology that enslaves us all (“we now have the technology to centrally plan agriculture so that there are no famines anymore” killed 100millions people in communist Russia. Some of them, due to famines ironically enough).
This is what is causing me to stop and consider: would I really get something out of using and understanding (read: trying the latest tools) the LLM/Agentic AI of today? I was bullish 3 years ago. Now much less so