Privacy Disasters: The Privacy CESspool
So many terrible ideas, so little time to rant about CES 2025.
Guess what, everybody?
The most overhyped tech show on the planet Most Powerful Tech Event in the World is happening right now.1 For those who remain blissfully ignorant somehow, CES — formerly known as the Consumer Electronics Show — is a four-day long masturbatory exercise of tech excess, hopium, and fawning journalistic praise. It’s been going in some fashion for almost 60 years, and every big tech player (and loads of upstarts, ~4,000 in all) all descend on the Nevada desert each January to share their vision, showcase their wares, and remind everyone that companies will continue to pitch solutions in search of a problem until the heat death of the universe.
Personal aside: In the late 1990s and early 2000s, I scored free tickets CES and a rival event, COMDEX (RIP) a few times.2 I remember being both overwhelmed and dazzled by all the raw possibility of the cutting-edge technology being promoted. This was of course, when tech companies were still innovative, and well before they became data-extracting vampires. Hell, Bill Gates even promised that everyone would be moving to the cloud within the next 10 years! In 2000! So much has changed.
Anyway. As is par for the course at CES, the press loves to chronicle all the shiny new things on display. Almost never critically, mind you —just non-stop gee-whiz, this-is-amazing praise. To be fair, there are lots of genuinely innovative, helpful things being reported at CES this year, especially in the medtech and mobility spaces, so it’s not entirely rage-inducing. I also wouldn’t mind having this next-level Pi-hole in my house. Ad and tracker blocking at the router? Sign me up. Also, this absolutely adorable cat that blows on a mug of hot coffee/soup to help cool it off is exactly the kind of thing that I would enjoy receiving as a birthday present. Assuming it actually becomes a thing and gets shipped outside of Japan.
But still, this wouldn’t be a privacy disasters post if there weren’t at least a few things that abjectly horrified me. So, buckle up, while I share a possible glimpse of future torment nexii we can all despair over in the next few years.
1. AI in Everything. Again.
Ever since ChatGPT became a household name, it’s been impossible for tech companies not to stuff an LLM (or ‘smart assistant’) into everything from refrigerators to toilets. But now we’ve got agentic AI which promises to do all the shitty things we humans hate doing (unlikely) whilst listening to all our conversations and reporting them back to data-hungry tech cos (considerably more likely). Or at least that’s how it’s being billed. However, despite the press coverage, most of the offerings being reported on thus far seem to fall squarely in the bog-standard chatbot camp. Por ejemplo:
Google wants to make AI-powered televisions even more of a thing by incorporating their comically bad Gemini LLM into new and existing Google TV sets. In addition to listening to everything you say, Gemini will also scrape news stories off the internet and summarize it for you. Users will also be able to ask Gemini for suggestions on what to watch, and what YouTube videos to check out.
Having used both Gemini and Google Assistant in the past, I doubt anything good will come from this unholy marriage. Aside from the inevitable copyright infringement suit from publishers, I suspect that anyone who buys these boxes will be greeted with loads of summarized ‘news’ from the likes of Reddit and The Onion, as well as unwanted recommendations on content. Users will also get much better at telling their TVs to fuck off, as Google plans to dispense with the wake word — meaning that your Google TV will be offering even more suggestions that nobody wants.Microsoft is threatening to cram Copilot not only in our PCs, but also TVs in a partnership with both LG and Samsung.
Samsung also believes that we all need smart assistant in our washers and dryers, nay, our entire homes! Think smart home tech on steroids, surely to tell us that our clothes are tatty and should be replaced. Oh, and they want to use ‘secure’ blockchain technology to protect all that data.
Oh dear god. Some company is releasing an AI-enabled Furby. And many others have various AI-enabled pets.
Why This is a Privacy Disaster
In addition to the fact that absolutely nobody needs AI-enabled appliances, the continued use of AI (agents or otherwise) are unlikely to incorporate any privacy-preserving features at all.
For example, it sems inconceivable that any of the processing and storage will be on-device, because the processing power necessary to support AI is much more than any smart washer, dryer, or TV is likely to manage. Instead, tech companies will shuffle even more personal data to servers in the sky. Or, in Samsung’s case, the blockchain.
And just think about all the juicy details that advertisers and the tech companies themselves will have access to —what food you have in your fridge, the state of your laundry, your TV viewing habits. All the gory data protection details — retention, data minimization, transparency, fairness, security — are left as exercises for the consumers to figure out. People are already worried that their phone’s AI assistants are spying on them 24/7 — so of course, why not spread that same concern to our entire home?
Oh, and if you’re hoping for regulation to save us… Guess again. Here’s a bit from the AP on what to expect from CES 2025:
The industry representing the companies involved welcomes government regulation. The government’s job is to say this is what is unsafe. And so the companies know what is legal and what is not. At the same time, a big challenge is if government stops innovation — so the question is for every country is finding that balance.
This is a portend to the fact that things will only get worse on the privacy & data protection front. Whatever contrition and pretense of companies caring will be flushed down AI-enabled shitter, as governments around the world take on a more authoritarian/fascist/anarcho-capitalist bent. By the way, if you’re playing along at home, the whole ‘regulation stifles innovation’ canard is a perfect example of a milgram question. Bonus!
And while this is not a privacy/data protection concern, we’re going to have to all think a little harder about what happens when these AI-enabled devices become obsolete. What happens to all those shiny AI/agents once the tech companies move on to the next sparkly thing?
2. Terrible Wearables & Literal Black Mirrors
CES 2024 featured lots of AI wearable hype, most of which did not pan out, including Spywearables like the Rabbit R1 and Humane AI pin (I wrote about Spywearables here).
Truthfully, I am already tired of wearables. Please make it stop.
A company called Halliday3 released yet another pair of smart glasses that feature content right in the user’s vision line. The technology claims to be able to analyze conversation and do live translation using AI, including fact-checking the person you’re talking with — making you the designated well ackshually guy at parties. Hilariously, the only mention of privacy on Halliday’s website is that the “display is only visible to you, ensuring complete privacy while remaining invisible to others.” Well done, lads.
Similar smart eyewear from companies such as Kickstarter-funded Mustard Glasses (ooof. Just try doing a Google/Brave search for that one). Mustard Glasses are somehow worse than Halliday, in that details of any kind are lacking, the website breaks, and it’s rife with spelling errors. I will be zero % surprised if this turns out to never ship.Smart rings are also becoming AI-enabled, and able to monitor bio-markers like sleep, stress, blood oxygen and “women’s health”. India-based Noise is leading that front, with its AI-enabled Luna Ring, made of, I shit you not, “Fighter Jet Grade Titanium”. This is the wearable version of ‘military-grade encryption’, clearly.
This is bad for many reasons. Firstly, I do not want to scream at my finger like a lunatic. Second, I do not need an AI-enabled menstrual health coach. What even is that? Third, I’ll explain my concerns about the whole women’s health angle below.Although it’s not a wearable, a bunch of companies, including Samsung, and French company Withings debuted full-body health screening mirrors. In Withings’ case, A literal Black Mirror, I might add:
Image credits: Withings
According to TechCrunch, this “‘conceptual product’ would offer 360-degree body scans, offering up user insight into weight, along with heart and lung health. Other metrics like sleep, activity, and nutrition would presumably come from a connected wearable”, presumably Withing’s ScanWatch. Oh, and there’s an AI assistant, to let you know if you’re a little too chunky, because of course.
There’s also a telemedicine portal that connects you to healthcare professionals, allegedly. While this is unlikely to be a thing (and would be cost-prohibitive for most people) it’s still deeply concerning in terms of privacy, especially as, again, nothing is stored on device and will end up in tech company hands.
Why This is a Privacy Disaster
Wearable tech, especially of the glasses / pins varieties, are a disaster on multiple fronts, assuming they launch or work at all. Firstly, there’s the obvious transparency and consent issues for any devices that are recording or processing the data of others. Take the Halliday glasses — in addition to recording audio, all those AI queries are being sent back to Halliday’s servers. I’m not sure where those servers are located, but the firm was founded in Shenzhen, China, so. And before people call me out, I’m also not particularly confident about data stored by most US companies under the new Glorious Leader, either.
Wearables in general are also fraught because they tend to be disproportionate data collectors & storers — both in terms of sensitive user data (mental and physical health, biometrics, sex life), as well as the data of anyone the wearer of said wearable interacts with. And that’s in addition to all the points I raised in this post.
Finally, at least in the US, wellness tech and wearables will continue to be dangerous for anyone with a uterus. Expect anti-choice states like Texas and Oklahoma to weaponize cycle and pregnancy data in an effort to further erode women’s rights and freedoms. Plus, so long as data brokers are permitted to sell data without consequence, I can see a thriving market for bounty collectors. As wellness tech discovers more about us, it may lead to consequences we haven’t anticipated.
3. Cities Straight out of Snow Crash
Yeah, so this last one isn’t unique to CES, but Toyota announced that their Woven City, a prototype city built on a 175-acre former Toyota factory near Mt. Fuji in Japan — will actually have people move in starting Fall 2025. Approximately 100 residents known as ‘Weavers’, will live in the Woven City, mostly consisting of a mix of Toyota employees, external scientists, entrepreneurs, and retirees. Eventually, the company hopes to accommodate around 2,000 people.
While the Woven City isn’t unique, it is the farthest along of the smart cities/company town hybrids. The Saudis’ planned linear smart city, Neom, isn’t expected until 2039 at the earliest. Meanwhile, Elon’s Starbase, is trundling along. Currently, around 200 SpaceX employees actually live there (in refurbished homes and Airstream trailers), but there’s no clear indication that this will be anything beyond a more traditional company town from the olden times. IDK. Maybe Starlink will be free?
Why This is a Privacy Disaster
To Toyota’s credit, a Privacy Notice for the Woven City exists, although it doesn’t have much in the way of substance. I haven’t been able to find any documents or other details approximating any sort of data protection or privacy assessment either. However, Toyota does acknowledge that the company will not provide residents’ personal data to third parties without consent, and that they are bound by Japan’s personal data protection laws and other relevant laws. It’s also plausible that Toyota performed reasonably diligent assessments of risks and designed features like mobility and the more ‘smart’ aspects of the city with privacy and data protection by design principles.
However, the data protection and privacy implications of these sovereign enclaves continue to be the kinds of things we discover after the bad things happen. My larger concern is that if Toyota’s Woven City is successful (and profitable), it may encourage more of these enclave / company towns. Even if Toyota does it right, the history of company towns generally is pretty awful, both in terms of curbing rights and freedoms of the inhabitants living there, and leaving people in precarity when one’s ability to live/eat/work is tied to a single entity.
Historically, these problems were bad enough — just imagine compounding all of that with constant surveillance, smart technology, company-mediated mobility, and social pressures, all managed by a ketamine-fueled megalomaniac billionaire.
Anyway, I think that’s enough. I’ve gotten myself off on a rant again, and now I need to drink.
BTW, did you know I was recently featured on the Masters of Privacy Podcast? You should totally check it out!
I pulled that first bit out of my ass, but I shit you not, ‘The most powerful tech event in the world’ is how the CES website actually bills itself. Also, despite this being the hugest most impactful tech event evar, I’ll note that as of the time of this writing, the ces.tech front page threw a 500 timeout error on me.
Upon reflection, I suspect that the reason I managed to score these otherwise quite expensive tickets was because I was a bit of a novelty — a real-life tech-savvy woman, hanging out on BBSes talking about hacking, phreaking, and other subversive things, as well as how to solder things and build a good gaming desktop, surrounded by guys often starved for female attention. Sometimes I wonder how I made it through my late teens and early 20s.
In true tech-dork fashion, the company is named after James Halliday from Ready Player One, because of course it is. Also, there is sweet fuckall about data protection or privacy anywhere on the website. Like, they’re not even trying.
Toyota is a WEF corporate partner for those playing at home:
https://www.weforum.org/organizations/toyota-motor-corporation/