Ford patents lip reading & emotion detection inside the car

Today's discussion focuses on a new Ford patent, but to understand its implications, it's crucial to review previous Ford patents and legal cases concerning privacy. One significant case from three years ago, in 2023, involved a judge ruling that automakers could legally download and store text messages under a Washington privacy law. This issue arises when a user pairs their phone to their car and grants permission for the car to read text messages aloud, a feature designed for convenience, especially to prevent drivers from looking at their phones while driving. The problem is that once the car retrieves these messages, it stores them internally, and crucially, there's no readily available way for the user to delete them. Car owners expressed concern that if they sell their vehicle, their stored text messages and phone data could be accessed by someone with the right software. The judge in this case ruled that the privacy law only applies if the user has suffered actual damages or reputational harm. This means that if someone accesses and stores your personal information from your car but hasn't yet used it against you, no privacy violation has technically occurred under that specific law. This creates a scenario where blackmail could exist without legal recourse until it's actively used, which is a serious problem. Another concerning Ford patent involved placing advertisements in cars. This patent detailed how an in-car advertisement system could monitor conversations to determine when and how many ads to serve, and whether to deliver them as audio or video, depending on driving conditions. This development is particularly troubling given the increasing cost and decreasing reliability of cars, suggesting that vehicles are not only becoming more expensive and less dependable but also actively spying on and advertising to their occupants. The most recent patent discussed is an "in-vehicle lip reading and facial expression detection patent application." This patent outlines how a vehicle could use acoustic signals, such as nearly inaudible sound waves, to analyze echoes from a user's lips and mouth, essentially employing active sonar on a user's face. The patent notably lacks any description of a consent process for opting into this technology. Furthermore, cameras in the vehicle could read lips and synthesize speech even if the microphone is muted or there's significant background noise. This "lip reading mode" would allow the vehicle to understand what a user is saying, even if their speech is inaudible or partially audible. The vehicle could then mute its microphones or the user's device and instead generate and synthesize speech based on lip reading and/or gesture detection, converting it into text or audio data to transmit to the other party in a conversation. This means speech could be transmitted as data without ever being acoustically recorded. Beyond just understanding speech, the car would also monitor facial expressions to infer emotional states, such as confusion or frustration. For example, if a user shakes their head or appears confused, the vehicle might determine they are having an issue with the communication system. This emotion inference is built into the system as a precondition. This is reminiscent of a Spotify patent that infers emotional states for advertising purposes, which is considered highly creepy and a reason to avoid such services. The patent also indicates that the control server for this system could be part of a cloud computing infrastructure and explicitly states connections to telematics service delivery networks, tow assistance firms, vehicle maintenance and repair firms, insurance firms, and transportation firms. This is particularly alarming because car manufacturers have been known to collect data from vehicles, sell it to brokers, who then sell it to insurance companies, leading to increased insurance rates. An example given is hard braking due to a drunk driver in front of you; the car's AI might simply record "hard braking" and send it to your insurance company, potentially raising your rates without understanding the context. This practice of manufacturers selling data to insurance companies to raise rates is not a conspiracy theory; it's a documented issue with ongoing lawsuits and has been reported by major news outlets. Additionally, the patent describes storing gestures and facial expressions in a per-vehicle database, associating them with corresponding verbal commands. This creates a persistent biometric database of a user's gestures and facial expressions tied to in-car commands. Crucially, the patent again fails to describe any mechanisms for data deletion, retention limits, or owner access. This reiterates the problem highlighted in previous court cases, where users lack the ability to delete their data from their own vehicles, and automakers and law enforcement agencies can retain and access this information indefinitely. While features like voice commands for air conditioning would be useful in modern cars lacking physical knobs, the current context is one where automotive manufacturers actively resist providing data deletion options, patent the ability to advertise based on in-car conversations, and sell user data to insurance companies without consent, leading to increased rates. This broader context transforms what might otherwise seem like a cool feature into a deeply concerning privacy invasion. It's important to note that these issues are not limited to specific brands like Tesla or electric vehicles. While Tesla often becomes a focal point for criticism, the reality is that all manufacturers, including traditional ones, are engaging in similar data collection, spying, and advertising patenting behaviors. Focusing solely on one brand risks making consumers of other vehicles less aware of these widespread practices. The argument that "just because they patent it doesn't mean they'll use it" is dismissed as naive, akin to having a detailed plan for nefarious actions but claiming no intent to execute it. The current situation reflects a disturbing trend where the privacy and data of vehicle owners are increasingly compromised across the automotive industry.