AI Seeks Out Racist Language in Property Deeds for Termination

Santa Clara County alone has 24 million property records, but the study team focused mostly on 5.2 million records from the period 1902 to 1980. The artificial intelligence model completed its review of those records in six days for $258, according to the Stanford study. A manual review would have taken five years at a cost of more than $1.4 million, the study estimated.

This is an awesome use of an LLM. Talk about the cost savings of automation, especially when the alternative was the reviews just not getting done.

Specialized LLMs trained for specific tasks can be immensely beneficial! I'm glad to see some of that happening instead of "Company XYZ is now needlessly adding AI to it's products because buzzwords!"
Given the error rate of LLMs, it seems more like they wasted $258 and a week that could have been spent on a human review.
- LLMs are bad for the uses they've been recently pushed for, yes. But this is legitimately a very good use of them. This is natural language processing, within a narrow scope with a specific intention. This is exactly what it can be good at. Even if does have a high false negative rate, that's still thousands and thousands of true positive cases that were addressed quickly and cheaply, and that a human auditor no longer needs to touch.
- What do you believe would make this particular use prone to errors?
Did you see something that said it was an LLM?
Edit: Indeed it's an LLM. They published the model here: https://huggingface.co/reglab-rrc/mistral-rrc
- Considering that it's a language task, LLMs exist, and the cost, it's a reasonable assumption. It'd be pretty silly to analyse a bag of words when you have tools you can use with minimal work with much better results. Even sillier to spend over $200 for something that can be run on a decade old machine in a few hours.

This is exactly the kind of task I'd expect AI to be useful for; it goes through a massive amount of freshly digitized data and it scans for, and flags for human action (and/or) review, things that are specified by a human for the AI to identify in a large batch of data.

Basically AI doing data-processing drudge work that no human could ever hope to achieve with any level of speed approaching that at which the AI can do it.

Do I think the AI should be doing these tasks unsupervised? Absolutely not! But the fact of the matter is; the AIs are being supervised in this task by the human clerks who are, at least in theory, expected to read the deed over and make sure it makes some sort of legal sense and that it didn't just cut out some harmless turn of phrase written into the covenant that actually has no racist meaning, intention or function. I'm assuming a lot of good faith here, but I'm guessing the human who is guiding the AI making these mass edits can just, by means of physicality, pull out the original document and see which language originally existed if it became an issue.

To be clear; I do think it's a good thing that the law is mandating and making these kinds of edits to property covenants in general to bring them more in line with modern law.

didn’t just cut out some harmless turn of phrase written into the covenant that actually has no racist meaning
I gotta say, because of the nature of systemic racism turns of phrase that are ambiguous or are explicitly neutral can be prejudiced or discriminatory is different ways.
We can't rely on a statistical model to tell us what is infringing on right. We have to be critical.

This actually isn't a terrible use of an LLM. It's actually kind of refreshing to see a news story about a beneficial use of it in a very specific context.

Next, ban SFH HOAs.

Could be a decent moderating tool too since increasing layers of Innuendo wouldn't be as likely to dodge a pattern seaking algoriðm as ðey would be an underpayed overworked hand sorting mod.

increasing layers of Innuendo
Well, also, these are documents written in the past, before 1948, when the Supreme Court invalidated the effect of racial covenants.
But the language remains, with no legal effect. But it's still there and should be eliminated. There's no cat and mouse game, just the need for cleanup of something left from the past.

I'm a bit confused. What have property records to do with racist language?