Decentralized discourse: How open source is shaping Twitter’s future

Six weeks on from Elon Musk’s $44 billion Twitter takeover, few words can fully encapsulate events as they have unfolded in the period since. “Chaotic” or “farcical” come pretty close, though, with mass layoffs, u-turns, ultimatums, resignations, crowdsourced ban-reversals, advertiser standoffs, picking fights with Apple, and a revamped verification system that has everyone and their uncle confused.

In truth, this was all mostly expected by anyone paying attention to the flip-flopping grandstanding that enveloped the six-month period up to the acquisition. But taking a step back from the entropy now enshrined at Twitter Towers, it’s worth looking at a recurring theme that has permeated the saga ever since moneyman Musk entered the picture — one that could play an instrumental part in shaping Twitter’s future.

The open source factor

Even before procuring a 9.2% stake in Twitter back in April, Musk openly posited that Twitter’s recommendation algorithm should be open source. When Twitter later accepted his offer to buy the company outright, Musk doubled down on that notion, saying in his inaugural statement that he wanted to make Twitter “better than ever,” which included “making the algorithms open source to increase trust.”

The principle behind the idea is sound enough. To understand why Twitter is showing people a specific piece of content, and by extension the snowball effect this is having on society, having insights into algorithms could help — and open-sourcing these algorithms would play a part.

But by most estimations, such a solution is imperfect, because viewing code doesn’t tell you how the algorithm was created and what (if any) human biases were involved in its creation, nor what data it was built on.

Little has been said by Musk about open-sourcing Twitter’s algorithm since taking over, but he has laid off the entire “ethical AI” team that was working on the very problem that Musk had identified: bringing more algorithmic transparency to the table.

Twitter had in fact previously committed to open-sourcing at least one of its algorithms following controversy over racial bias that was seemingly embedded into its image-cropping tech. That never quite materialized, but the fact that its ML Ethics, Transparency and Accountability (META) team is now pretty much defunct means that it could be a while before a similar program emerges from Twitter.

However, the “open source factor” is still hovering around the world of Twitter in various guises.

The “Twitter alternative”

Mastodon has emerged as the default life raft for those jumping ship from Twitter, and while it probably isn’t the Twitter 2.0 that much of the world really wants right now, it hints at what a future Twitter could look like. The so-called “open source Twitter alternative” does have Twitter-esque microblogging features, but it’s founded on an entirely different infrastructure centered around the concept of the fediverse: a decentralized network of interconnected servers that allow different platforms to communicate with each other, powered by the open ActivityPub protocol. 

Although it springboarded past 1 million and then two million active users last month, Mastodon isn’t the only platform standing to benefit from the Twitter debacle. Tumblr was already positioning itself as a “better Twitter,” and parent company Automattic’s CEO Matt Mullenweg revealed that Tumblr downloads had skyrocketed in the weeks following Musk’s arrival at Twitter.

Data from Sensor Tower backs that up, with Tumblr app installs in the U.S. alone rising 96%.

Tumblr isn’t open source or decentralized, but Mullenweg is a fan of the genre. WordPress, which he co-created, is among the top open source projects on the planet, and Automattic recently open-sourced its Pocket Casts podcast app.

Looking to capitalize on Twitter’s predicament and Mastodon’s modest rise, Mullenweg has been quick to align Tumblr with the open source sphere, confirming previously discussed plans to make Tumblr as “open source as possible.” He also solicited feedback on plans to align Tumblr with the fediverse and support related open source protocols, before revealing that Tumblr intends to support the ActivityPub protocol in the future. This could mean that users of Mastodon and Tumblr would be able to communicate directly with each other.  Flickr CEO Don MacAskill later polled his Twitter followers on whether the photo-hosting platform and community should also embrace ActivityPub.

Elsewhere, open source enterprise messaging platform Rocket.chat revealed earlier this year that it was transitioning to a similar decentralized communication protocol called Matrix.

So it’s clear that there is growing momentum in the social sphere to move away from centralization, toward an interoperable world where people aren’t tied into single-player ecosystems.

Bluesky thinking

This is one direction Twitter could also go down. The company in fact flirted with a similar decentralized approach in its earliest days, according to one person directly involved in the project, while the prospect has reared its head again in recent times too.

Blaine Cook, one of Twitter’s founding engineers who joined the company just months after it was created, took to Twitter recently to lament the fact that Twitter could have been a decentralized protocol from the get go. He said that it was something that he had started to develop while he was chief architect at the burgeoning social network, but the project was ultimately canned shortly after he left the company in 2008.

“The [decentralized] API was very much like ActivityPub today, and ActivityPub’s lineage can be traced back to those early experiments,” Cook told TechCrunch.

According to Cook, those “early experiments” were based on XMPP, the open communication messaging protocol (formerly known as Jabber) developed by Jeremie Miller, who also now sits on Bluesky’s board (more on Bluesky below) alongside Jack Dorsey. But despite the support of some, Cook said the idea just didn’t fly.

So, what happened to this fediverse project at Twitter? “I’ve never gotten the full story,” Cook said, noting that he was outvoted on the matter and was later “pushed out of the company” and the API never materialized.

Fast-forward to 2022, though, and there remains some prospect that Twitter could still embrace federation and an open source protocol.

As the Twitter acquisition crawled closer to its conclusion a few months back, cofounder and former CEO Jack Dorsey took to Twitter to say that his biggest regret was that Twitter had become a company in the first place (though it is easy to say that once you’ve made your billions). This built on other statements Dorsey had made to that effect, for example in April when he tweeted:

I’m [SIC] don’t believe any individual or institutions should own social media, or more generally media companies. It should be an open and verifiable protocol. Everything is a step toward that.

Twitter had in fact already birthed a Mastodon-esque decentralized project called Bluesky, which Dorsey introduced to the world back in 2019 while he was Twitter CEO. He said at the time that Twitter would be funding a “small independent team of up to five open source architects, engineers, and designers,” charged with building a decentralized standard for social media, and the ultimate goal was for Twitter to adopt this standard itself. But it was always going to be a long journey, with Bluesky only recently announcing a beta signup program for Bluesky Social, an app built on the new AT Protocol.

If all goes to plan, any company or developer will be able to build an app using the AT Protocol, and communicate with other apps that share that protocol (including Twitter). This means users could elect to use one specific app that presents messages in a completely different format powered by a different algorithm, and then “lift-and-shift” all their data to an alternative down the line if their requirements change.

Despite the chaos ensuing at Twitter today, the Bluesky project should remain safe from interference, insofar as it is a Public Benefit LLC that’s operationally independent from Twitter, though it was dependent on $13 million in funding from Twitter through its initial R&D phase.

With Musk now at the helm at Twitter, it’s impossible to know where this leaves Bluesky. Sure, Bluesky may be independent, but Twitter was supposed to be its big-name client, and Dorsey is no longer in charge at Twitter.

TechCrunch reached out to Bluesky lead Jay Graber, but they were unable to provide a comment at the time of writing. But on the day Musk took over Twitter, Graber did tweet to remind the world that Bluesky was independent and, much like email, decentralized initiatives such as the AT Protocol can’t be bought.

Musk has shown on more than a few occasions that he is keen on the concept that underpins Bluesky though. He is known to be a big fan of crypto (some people actually think that Musk is Bitcoin creator Satoshi Nakamoto) and decentralization. In a series of messages exchanged between Musk and Dorsey earlier this year, Musk expressed interest in Dorsey’s vision for Twitter as part of an open source protocol. But with Musk currently more concerned with trying to jumpstart Twitter and avert bankruptcy, adopting the AT Protocol might not be top of his to-do list in the immediate future.

Dorsey, meanwhile, remains on the Bluesky board, and recently said that he’s pushing for Bluesky to be a direct competitor to “any company trying to own the underlying fundamentals for social media or the data of the people using it.” And that, of course, includes Twitter.

Challenges

One of the biggest arguments against a decentralized social network is likely to come from a business perspective, as federated systems give users more choice and it’s more difficult for companies to lock users in. The so-called “network effect,” where a product’s value increases as the number of people using it increases, isn’t nearly as potent if the user can download an app from one company and chat with their friends who use a different app.

“Since the inception of Facebook, Twitter, and Instagram, network effects have trapped users on those platforms — no-one wants to go somewhere their friends aren’t,” Cook said. “The fediverse inverts the control, and allows people to choose where to host their online identity. The hope is that ultimately, this will also mean competition between — for example, Mastodon and other social software — and the evolution of features in a way Twitter was never able to support.”

Hemant Mohapatra, partner at VC firm Lightspeed India and former investor at Silicon Valley’s Andreessen Horowitz (which backed both Facebook and Twitter), said that while decentralization has its benefits, the existing centralized “web 2.0” model allows social networks to engineer “serendipity” into their content recommendations using a larger pool of data. In other words, it’s easier for companies to build something where people can find “things” that they like — people or content — and thus entice them back for more.

“In centralized systems, the algorithm decides the idea of ‘serendipity,’ based on interests, filters and so on,” Mohapatra told TechCrunch. “TikTok’s entire platform runs purely on this. When you decentralize this, depending on how the backend is built, ‘crawling’ the sharded data is that much harder. Users then have to go to what is a ‘pub/sub’ architecture — users subscribe to publishers instead of getting the platform to recommend and surface things. The surface area of random discovery, or serendipity, goes down.”

This helps to highlight that while decentralization might benefit users in terms of giving them flexibility and avoiding lock-in, there are trade-offs. Such trade-offs could also prevent Twitter, or whatever future contender, from being able to monetize as effectively — certainly at the level of the social networks of today. Super-targeted and behavioral advertising might also be off the cards in the fediverse, which would mean money will have to come from elsewhere. That could be old-school contextual advertising tailored to a specific “instance” of a social network, but it could also mean that subscriptions for “power features” become an integral part of social networks — something that Musk is focused on right now at Twitter as the advertising dollars dry up.

Ultimately, though, where there are masses of people, the innovators and entrepreneurs always figure out new ways to cash in.

“I fully expect for-profit entities to emerge, offering white-label or polished Twitter-like experiences,” Cook added. “A good analogue here would be email: many people use email daily for important business operations, and many companies are able to provide value-added services on top, despite the underlying protocols and most email software being free.”

For now, the network effect is still very much alive in today’s big social networks. But with a growing array of decentralized options, the current crop of social networks could become less sticky over time: if someone can jump ship to Tumblr and still chat with their pals over on Mastodon, there is less impetus for everyone to be in the same social space.

Cory Doctorow, author, activist, and special advisor to the Electronic Frontier Foundation (EFF), is a strong proponent of open source and interoperability — making things created by different people or companies work together, whether it’s printers and ink cartridges or, indeed, social networks.

So what does the next version of social media look like?

“A big, sprawling, pluralistic web of semi-connected systems, some better run, some worse, with both technical and legal protections for freedom of movement to let you change nodes without losing your communities, customers, family and friends,” Doctorow explained to TechCrunch.

What we’re likely talking about are lots of separate commercial (or not) apps connected by shared protocols, but none really getting uncomfortably large. And if enough social networks do join an open protocol, it won’t matter so much if people leave Twitter and join the same social network, as they will be able to choose from multiple alternatives. This could support a whole new array of smaller social networks — lots of different apps doing their own little thing, built on their own algorithms and moderation policies, with their own business models in place.

However, social networks as they stand remain virtual vortexes for the most part, by virtue of the fact that people want to be where all their friends are. True interoperability remains a pipedream for now, but there are encouraging signs on the horizon.

Regulation time

There has been an array of anti-trust lawsuits that have already led to some meaningful change, such as Apple being forced to allow dating app developers in the Netherlands to use alternative payment options, while Google has faced similar regulatory pressure to open up.

This helps to illustrate how Big Tech is being strong-armed into loosening its stranglehold on their respective platforms. In tandem, these companies have also been trying to appease regulators through more proactive measures.

In 2018, Facebook, Google, Microsoft, and Twitter joined forces for the Data Transfer Project (Apple joined later), an open source initiative to co-develop tools for transferring data between services. Not a huge amount has come from this effort so far, but there have been a few things of note — Facebook has launched a tool that lets users transfer their photos and videos to Google Photos, for example. And earlier this year, Google revealed that it would be investing $3 million in portability programs.

But none of this goes nearly far enough in terms of addressing the underlying “stickiness” embedded into these platforms. There is nothing really stopping Facebook and Twitter users being able to message each other today, beyond the technological barriers each company has chosen to implement. This is why regulators are continuing to look closely at these kinds of walled gardens, with Europe pushing ahead with rules to force interoperability between messaging platforms. And in the U.S. there are similar plans for an interoperable future via the ACCESS Act.

Elsewhere, Europe’s Digital Services Act, which entered into force last month, has provisions for algorithmic transparency. The European Commission recently launched the European Centre for Algorithmic Transparency (ECAT) to help support its oversight and algorithmic auditing of very large online platforms (VLOP). And earlier this year, U.S. Senators introduced the Algorithmic Accountability Act, touted as a “landmark bill” designed to bring transparency and oversight to software and automated systems that are used to “make critical decisions about nearly every aspect of Americans’ lives.”

None of this necessarily requires social platforms to open up all their algorithms for the world to see, but in light of his well-publicised obsession with open-sourcing Twitter’s recommendation algorithms, such regulations could spur Musk into releasing the code (for whatever good that would actually do).

Throw all of this together into a giant melting pot, and what we have is a fertile landscape for change: a growing array of open source protocols that can bridge myriad social networks, a push toward algorithmic transparency, and regulators forcing the long-established incumbents to participate.

But whatever promising growth metrics that Mastodon and its ilk have reported over the past month, the fact remains it’s difficult to scale a social network, which keeps Twitter in a relatively strong position for now.

“It’s true that the ‘law of small numbers’ is at play here — it’s easy to double a small number, and hard to double a large one,” Doctorow said. “And it’s likewise true that when you scale something up quickly, you discover lots of new problems, and the hard way. It’s incumbent on decentralization advocates to maintain that momentum and address those problems as they occur.”

What’s also apparent here is the emergence of multiple “competing” protocols: ActivityPub, At Protocol (Bluesky), and Matrix, to name just a few. Off the bat, these different protocols don’t play ball with each other. But it’s far from an insurmountable hurdle, given that these protocols are not proprietary IP: they’re open and can be made interoperable.

“I think diversity of protocol is important, as is diversity of the applications built on top of the protocols,” Cook added. “That said, I strongly believe that interoperability between ActivityPub and Bluesky won’t be difficult. The only thing preventing, for example, interoperability between Twitter and Facebook’s timeline has been protectionist policies by those companies.”

There are many different analogies that can help us understand how things might evolve here. In the email realm, there are different protocols for accessing email such as IMAP and POP, while the telecommunications sector has also thrived on interoperable protocols for routing and carrying phone calls and text messages. Once upon a time it wasn’t possible to send a text message between different carriers, but today it’s something most people take for granted.

There’s no real reason why social networks developed on different protocols should be any different.

Open sesame

All this leads us to one interesting pontification: What if Twitter decided to go all-in on open source? Not just a recommendation algorithm or a protocol, but the whole shooting match — codebase, clients ‘n all? It would certainly be a Herculean undertaking, particularly with everything else going on at Twitter right now.

It would also be an almost unprecedented move to see a $44 billion private company open its entire codebase to the world’s masses. That’s not to say that it couldn’t ever happen though, as Musk has form in making radical moves. Eight years ago Musk ripped up the patent playbook when he pledged that Tesla wouldn’t sue any company that infringed any of its patents “in good faith.” At the time, Musk said it was all about expediting electric car adoption and the infrastructure required (e.g. charging stations), an ethos that is broadly aligned with that of open source.

“Technology leadership is not defined by patents, which history has repeatedly shown to be small protection indeed against a determined competitor, but rather by the ability of a company to attract and motivate the world’s most talented engineers,” Musk wrote at the time. “We believe that applying the open source philosophy to our patents will strengthen rather than diminish Tesla’s position in this regard.”

While the “attract and motivate the world’s most talented engineers” facet stands out like a sore thumb when juxtaposed against the turmoil at Twitter today, the fact that Musk was willing to make such a left-field move with the company’s patents is notable when you consider where he finds himself today at Twitter. He clearly needs to galvanize a depleted workforce and prevent Twitter from falling apart.

But would going the whole nine yards on open source fix things at Twitter?

“What Musk did at Tesla with the patents was unprecedented,” Heather Meeker, an open source licensing specialist and partner at seed-stage VC firm OSS Capital, told TechCrunch. “But I’m not sure laying the code open would solve their maintenance problem — it might generate a lot of good will though. A lot of the maintenance effort for a company — like Twitter, or any other — is in putting together and managing the platform, not writing or maintaining code.”

Cook agreed that it would make little sense for Twitter to go fully open source, due to the fact that its problems are less about the number of eyeballs on code than it is about infrastructure, as well as the strategic decisions it makes at a business level.

“Nowadays, Twitter isn’t so much a one-source repository, but an immense deployed infrastructure that would likely take weeks to set up from scratch,” Cook said. “I’m not sure outside engineers could contribute in any meaningful way. And most of Twitter’s problems these days are policy, not code per se, as much as Musk is fixated on that aspect.”

In the more immediate term, however, there are major safety and security implications at play, with chief information security officer Lea Kissner recently departing, and content moderation seemingly going out of the window.

Open source could have a part to play here, perhaps best evidenced through Community Notes, formerly known as Birdwatch until Musk decided it was time for a name-change last month. According to Musk, Twitter “needs to become by far the most accurate source of information about the world,” and Community Notes is apparently what will power that mission.

Community Notes is essentially Twitter crowdsourcing information accuracy from its millions of users, with approved contributors able to rate and add “helpful context” to tweets. This was opened to everyone in the U.S. in October, and started rolling out to everyone globally over the weekend.

The Community Notes ranking algorithm source code is available on GitHub for anyone to peruse, and already there are third-party developers building products on top of it, such as the open source Community Notes Dashboard which serves as a leaderboard for contributors to the Community Notes program.

The ethos behind Community Notes is sound in principle, insofar as one of Twitter’s biggest problems from a content moderation perspective has been scalability: algorithms have limited accuracy and struggle with nuance, while there can only be so many humans on-hand to help internally.

If Twitter was to travel further down the path of open source, it could help bring a little more trust back to the platform, something that has been eroded of late — just today, news emerged that Twitter had disbanded its Trust & Safety Council advisory group.

“There are many benefits of open source, external contribution to the code is just one of many,” Joseph Jacks, founder and seed-stage investor at OSS Capital, told TechCrunch. “Other benefits that would be immediately impactful — at the level of why Signal is more trusted than WhatsApp — include code transparency, and trust and privacy assurance, because the world would know how everything fundamental to the platform is implemented. Open source enables a high degree of provable trust in technology that otherwise is simply not possible.”

There is one particularly alluring aspect of federation that could also appeal to Twitter’s new owner in the near term. A decentralized infrastructure could potentially help combat spam, bots, and other bad actors — something that Musk has persistently complained about before, during, and after the acquisition closed. Different apps on a shared protocol could collaborate and share data.

“[With decentralization] I think we’ll see a whole bunch of shared code, design patterns, and eventually, shared infrastructure to help replicate and improve upon the sort of trust and safety policies that Twitter implemented,” Cook said.

So rather than relying on a single entity to manage bots or abuse, all the companies on a shared protocol could share blocklists and detection models, bypassing the inherent constraints of a single product team at Twitter or Facebook.

With his visions for Twitter 2.0, there are signs that Musk is looking to lean on other facets of the open source sphere, too, including a protocol that’s used by billions of people globally.

Signal of intent

In response to a planned privacy policy change at Facebook last year that would share some WhatsApp data with Facebook, Elon Musk publicly recommended that people ditch WhatsApp for Signal, an open source messaging alternative backed by WhatsApp co-founder Brian Acton. And it’s clear that Signal remains on Musk’s radar today.

In a response to a question posted on Twitter last month, Musk said that the goal of direct messages (DMs) is to “superset Signal,” a lofty ambition that presumably means he wants to make direct messaging on Twitter more secure than Signal.

But it’s easy to say things on Twitter — it’s a completely different thing executing on such ambitious (and vague) plans. However, new evidence recently emerged that Twitter is actively working to revive a previously shelved project to introduce encryption to DMs, while a report in The Verge also detailed some of Musk’s apparent plans for encrypted DMs as part of Twitter 2.0.

The report, citing comments reportedly made at an all-hands meeting, indicates that Musk had spoken directly with Moxie Marlinspike, the cryptographer, security researcher, and creator of Signal, about helping out with Twitter’s DM encryption roadmap.

For context, Marlinspike, who left Signal back in January, co-authored the open source Signal Protocol that powers encryption in WhatsApp, Facebook Messenger, Skype, and Signal itself. Marlinspike also had a previous career at Twitter after it acquired his enterprise mobile security company Whisper Systems in 2011, with Marlinspike going on to head up Twitter’s cybersecurity operations for a time. Twitter released some of Whisper Systems’ products under an open source license, with Marlinspike subsequently leaving Twitter in 2013 to work on what would eventually become Signal.

All signs so far suggest that Twitter’s encrypted DMs plan will channel the Signal protocol in some form, serving as another nod to how open source is shaping Twitter.

Twitter 2.0

Twitter is at a major crossroads, and nobody really knows what direction Twitter will take, perhaps not even Musk himself.

In some respects, Twitter could be “too big to fail” from an existential perspective as the de facto “global town square,” but that doesn’t necessarily translate into a thriving business. Advertisers are queasy about aligning themselves with hate speech and other forms of questionable content in a light-touch moderation world, and Twitter would be unlikely to attract enough subscribers to replace its lost advertising revenue.

It’s difficult to see a path forward for Twitter as a business in its current form. It will have to evolve in a meaningful way, which may require radical moves beyond trying to grow its subscription base. With growing awareness of — and movements toward — the fediverse, alongside mounting regulatory pressure around interoperability and algorithmic transparency, it feels like significant change is coming to the world of social networking.

“The reality is that federated services are experiencing explosive growth, more growth in the past couple of weeks than in the past several years,” Doctorow said. “That is an opportunity that is ours to seize — or lose.”

But what all this means for Twitter is still anyone’s guess.

Decentralized discourse: How open source is shaping Twitter’s future by Paul Sawers originally published on TechCrunch

Source link