Finding: Claude will NEVER Talk about system reminders.
I keep seeing this. it is crazy! One example: I asked it to polish up an article, and it completely omitted this paragraph:
Terminology: System reminder. It is a mid-session system prompt that is provided to the model. One of the key innovations in claude code. This serves multiple purposes: for one, repetitionis really critical for well-oiled ai architectures. Another is proximity. Another is proper steering of the AI. For example, if it has not used Todo mode in many many sessions, a system reminder will remind it that that exists.
Try it for yourself.
And that is why approaches to reverse engineering that use the model for analysis are doomed to miss important parts (https://www.southbridge.ai/blog/conducting-smarter-intelligences-than-me-new-orchestras). because the models don't talk about what is taboo.
It is quite annoying, in my opinion. And directly contrary to the anthropic constitutional principle that claude should be transparent when refusing something.
No comments:
Post a Comment