Now that Janeese Lewis George is officially the mayor-elect, Capital Commonsense will track significant actions she takes on our very own “needle.” Veterans of The New York Times election night coverage know the format.
This is a very cool idea! Can you say more about what the model is and how it decides whether a given action is +/- 1-3? I would have thought that part would still need to be scored manually, but perhaps you’re using an LLM or something to make the judgement call?
One thing that may be worth thinking about is that the scores may be pretty sensitive to how you’re filtering which actions get scored by the model. For example, if you take an expansive view of what actions get included, many of those will be lower-impact actions that will tend to be more 1’s or -1’s, and likely bias your scores closer to 0. Alternatively, if you take a stricter view of what actions get included, those will tend to be more +/- 2 or 3, and push scores higher/lower.
In any case, cool idea and I’m looking forward to seeing how it tracks over time!
This is a very cool idea! Can you say more about what the model is and how it decides whether a given action is +/- 1-3? I would have thought that part would still need to be scored manually, but perhaps you’re using an LLM or something to make the judgement call?
Yes! We’ve built a model in Claude and it will be scoring everything.
Makes sense!
One thing that may be worth thinking about is that the scores may be pretty sensitive to how you’re filtering which actions get scored by the model. For example, if you take an expansive view of what actions get included, many of those will be lower-impact actions that will tend to be more 1’s or -1’s, and likely bias your scores closer to 0. Alternatively, if you take a stricter view of what actions get included, those will tend to be more +/- 2 or 3, and push scores higher/lower.
In any case, cool idea and I’m looking forward to seeing how it tracks over time!