Citizendium struggles to convert publicity into content

February 14, 2007

Citizendium was slashdotted again recently, prompting the creation of 900 new user accounts in the last 24 hours. So, following up from my recent post on their last slashdotting, how has this affected the site’s performance?

Vandalism at Citizendium

Let’s cut right to the chase, here: how many people were blocked shortly after joining, as vandals? Let’s compare the 899 new users for the last 24 hours with a quick look at the IP Block list at the pilot, which reports that 108 users were blocked in the same time period; this drops the number of new users down to 791, which is still an impressive enough figure. On the other hand, a 12% block rate is not impressive, and may produce major problems if it isn’t corrected in the future.

The figures also indicate that the rate of vandalism at Citizendium is much worse than it was the first time around:

Not only has the number of users spiked considerably, but the number of blocked accounts has done the same. Considering that the number of users capable of fighting vandalism has remained largely the same throughout, this represents a significant increase in those individuals’ workloads.

One point that I should make about the above graph is that the number of user accounts created on the 15th and 16th of February is very difficult to determine, because starting at 17:14, 15 February 2007, Citizendium was hit by an account creation bot that created approximately 11,000 dummy accounts before it was stopped — nineteen hours later. Call me skeptical, but that’s not the kind of lightning-fast emergency response that Citizendium really needs.

Here’s a screenshot of the activity, which was officially acknowledged by Citizendium’s Jason Potkanski today.

Building the encyclopedia

Now, how many contributions are the 791 new users helping with? The recent changes list at Citizendium tells us exactly how many edits have been made to articles in the last 24 hours, and combined with the graphs from my last post on the topic we have a wider picture of how the project is coming along (full size):

The conclusion for the current slashdotting will take a few days to become clear, but it’s certain that the previous one produced little change — if any — in the rate at which the encyclopedia expands. Despite the good publicity it’s been receiving, it appears that little has changed.

One possible reason for this was put forward by Aaron Schwartz in “Who Writes Wikipedia?“, in which he argued that the vast majority of edits (spelling and grammar corrections, formatting, and so on) came from a small core of registered users, but that the majority of the site’s content came directly from anonymous users.

When you put it all together, the story become clear: an outsider makes one edit to add a chunk of information, then insiders make several edits tweaking and reformatting it. In addition, insiders rack up thousands of edits doing things like changing the name of a category across the entire site — the kind of thing only insiders deeply care about. As a result, insiders account for the vast majority of the edits. But it’s the outsiders who provide nearly all of the content.

This has some serious implications for Citizendium. If content is ultimately produced by anonymous users, then refined by the few that register, Citizendium’s enforced registration could present a significant barrier to its own growth; the “real names policy” that the site also attempts to enforce adds to that, by requiring that normally-anonymous users divulge more information about themselves than they may be willing to.

Conclusion

The past two slashdottings at Citizendium have gone a long way to demonstrating that more is needed to reach Release Day than just good publicity. Even with these two appearances at Slashdot in less than a week, and numerous other articles besides, the number of article edits per day has hardly budged. Indeed, the number of edits we saw after the first slashdotting was fewer than just a few days before it, underscoring the fact that good press does not translate directly into new content.

As for the current appearance at Slashdot, I expect the result to be much the same as with the last. While it’ll take a few days for the data to become available, my prediction — and I think it’s a very safe one — is that Citizendium will ultimately be unable to convert the thousands of interested people flocking to their site into new contributors without significant changes to their editing policies. The real names policy and enforced login, for a start.

Technorati Tags: , , ,

Advertisements

13 Responses to “Citizendium struggles to convert publicity into content”

  1. Mike Johnson Says:

    Davey,

    I think you have a choice to make. You can choose to write hit pieces on things you dislike and be an internet jackal– spotting, feeding on, and exacerbating others’ weaknesses, and generally making the world a shittier place– or or you can use your talents to make the world a better place. Make something. Improve something. Etc.

    I wouldn’t mention this if I didn’t think you had the potential to do good things that can make a difference– you’re a smart, motivated fellow– you just need to make the choice not to be a jackal and to do them.

    Mike Johnson

  2. Otto Kerner Says:

    He came up with info about recent usage patterns on Citizendium. That seems useful to me.

  3. daveydweeb Says:

    Mike, as right as you are to point out that I’m currently very (over?) interested in the progress of Citizendium, it’s hardly fair to call me a jackal for the above reporting. Was I supposed to hold my tongue when I noticed the trends I reported on, or am I a jackal for deciding to post about them?

  4. Mike Johnson Says:

    David,

    I think you do really great investigative analysis- your data’s interesting and well-formatted- and (as you may guess :) I’m receptive to the argument that requiring that people log in to edit may turn out to be a significant hurdle. I don’t agree with your general conclusion that “Citizendium fails to convert publicity into content” – I think it’s much too early to make any sort of conclusion there.

    Getting users to contribute to a community site is difficult, yes, but the process can also be prolonged and hard to evaluate, as many times users will create accounts, bookmark a site, and come back later when they’re in the right frame of mind to engage the site. I think the ‘edits per day’ metric also has its weaknesses, as Aaron Swartz’s essay you quoted from argued that not all edits are created equal, and in fact, the vast majority of edits at Wikipedia (and by implication Citizendium, though it’d be difficult to say for sure) involve rather trivial changes, whereas a minority involve large content contributions. This makes a ‘number of edits’ metric less useful, as it adds to the already-significant statistical noise, and it’s quite possible that the ratio of minor-to-major edits is different on high-traffic days. Similarly with vandalism, there are multiple reasons for blocks: most happen because of users creating names that don’t follow our naming standards, not because of vandalism (I’d have to check, but off-hand I don’t remember any vandalism from the last Slashdotting).

    So, though I don’t agree with your conclusions here, one thing I really like about your blog is you try to tackle relevant things. You’re drawn to relevance. That’s a great quality, and I hope you realize how rare it is.

    However, where I was coming from with the ‘internet jackal’ remark is that you seem to have made up your mind on Citizendium- you don’t like it- and have made it your goal to undercut it on several different levels. That may not be fair, but judging by the venom in your recent posts about Citizendium (i.e. hubris, horses, and arguably leaping to a poorly-supported negative conclusion in this post), it certainly can’t be ruled out. And if that’s the case, your blogging makes the world a worse place.

    I have trust that this isn’t the case. That your blogging about Citizendium is motivated by more lofty factors than spite and the desire that Citizendium fail. I’d love to see (and expect) deeper, more balanced posts here that go beyond surface evaluations and speak to the strengths *and* weaknesses of different wikis and wiki models.

    Prove me right, eh? :)

  5. daveydweeb Says:

    Mike, I disagree with your opinion on the “edits per day metric”. In general terms, you’re quite correct: the number of edits made to a project in a given period of time can’t really be used as an accurate indication of its progress.

    *However*, we’re talking about a project that was only recently Slashdotted *twice* in the space of a week, and once more several months ago. The tens of thousands of people that would have descended upon the site in the space of a week produced over a thousand new users in the same space of time — and yet the rate of contributions remains roughly steady over time. There are two conclusions to draw from this: that these new users are not contributing, or that the overhead produced by so many new users means that CZ’s core contributors are unable to continue in their original line of work.

    Now, I’d like to take this information further, but that’s impossible for me to do: take a look at Special:Newpages at Citizendium, and note that it lacks timestamps like at RecentChanges.

    However, the fact that such extremely positive publicity has failed to change anything but the rates of account creation and account blocking, tells me that Citizendium needs to do more to encourage these newbies to contribute. Larry actually seems to be doing quite the opposite at the moment, based on this choice quote from his recent “how to convert Wikipedia articles to Citizendium articles” essay:

    “Have you actually bought into this Citizendium thing at all? If not, leave well alone, please, and stick with Wikipedia until you’re actually convinced.”

    The above paragraph reads very much like, “casual contributors, shove off”, effectively attacking the very group of users that promise to produce the most content for Citizendium. That very stance is common at CZ, and badly needs to be addressed.

    As to my blogging, honestly, I’d post about the strengths of Citizendium if I could think of any that were worth mentioning. As it is, what advantages it may have — limited vandalism, real names, expert help — are negated by inexperience in dealing with vandalism, an ineffective means of identity checking, and an Author-Editor-Constable editing model that alienates casual users and essentially gives “experts” a free ride.

  6. Mike Johnson Says:

    I would suggest that we have had more article edits of late– 130 on the day prior to the slashdotting vs 308 post slashdotting, by your numbers. But this is rather secondary to my main point.

    My main point is “that you seem to have made up your mind on Citizendium- you don’t like it- and have made it your goal to undercut it on several different levels.” I hope your blogging about Citizendium is motivated by more than the jackal’s desire to see Citizendium fail– but that’s not obvious by your recent posts. Practically speaking, there are good things in your posts (as I’ve said before), but as advice it completely fails, because people won’t listen to someone who insults them. It doesn’t come across as constructive criticism, merely venomous, hateful criticism which serves no constructive purpose.

    There are a lot of haters, naysayers, and jackals in this world; most of them are so because they can’t do anything else. You can.

  7. Larry Sanger Says:

    David, what you have said is literally false: “Citizendium fails to convert publicity into content.” We did convert publicity into content; as Mike pointed out, we increased our number of contributions and over the long haul, no doubt, will have benefitted from it. I’m puzzled why you go to such lengths to put the veneer of statistics on a claim that is quite straightforwardly false. You’re increasingly looking like a “troll” and thus someone with no credibility.

  8. daveydweeb Says:

    Larry, take a look at the graphs above. As with the first slashdotting of the month, the contributions of your new users are again tending towards zero deviation from the norm — you’re looking to be left exactly where you were before, minus the 1-200 additional contributions you saw during the high.

    Now, I’m guessing your definition of “failure” is somewhat different to mine. You’ve literally generated an additional few hundred contributions during the short high following your slashdottings, but those numbers are so small as to be meaningless, a statistical blip. A pedant would call that success, but it’s a stretch of the imagination.

    By the way, Larry, would you like to comment on the nineteen hours it took to block the account creation bot? Even Wikipedia is better than that.

  9. Larry Sanger Says:

    As is usual with trolls–not even an attempt to respond to the very, very basic point I was making. Your title is a lie, and your analysis plainly sophomoric for reasons Mike pointed out, and others besides. You also failed to engage meaningfully with Mike on his central point. David, you’re wasting everyone’s time. Over and out.

  10. daveydweeb Says:

    I’ve changed the verb, Larry, just for you.

  11. Varsha SIngh Says:

    Hmm…I’m the person that wrote and ran the vandal account creation bot. Mostly because I was offended by Larry’s asshole-ness, elitisim, criticism of Wikipedia’s “anti-elitism” (which I prefer to call what puts “free” in “The free encyclopedia”, his intention to “unseat it”, and his overall hunger for power.

    Just a few corrections (and nice blog, by the way!): It didn’t really take them 19 hours to block my IP. In reality, I have a dynamic IP, so when they blocked it (which generally took between 3 and 10 minutes), I just restarted my router and got a new IP. Also, you estimate that 11,000 accounts were created. I think that’s a low statistic. When I started the mass creations, Special:Statistics was putting out 6,043. Now, it’s somewhere around 30,000. I seriously doubt that 16,000 valid accounts have been created since a few days ago, so I assume that 99% are mine…you do the math.

    I don’t know if I will vandalize the CZ again. While I’m still disappointed with what I cite above, I don’t think CZ needs my help to fail. Larry is doing a wonderful job of fucking it up all by himself. Also, I’m pretty busy making productive bots for Wikipedia. (Yes, it’s actually harder to make a productive bot than a vandalbot :-D)

  12. BD Says:

    Citizendium was created to prove that S was right, so a failure on its own is much more interesting.

    ‘Have you actually bought into this Citizendium thing’ on yes or no basis…Looks like he wants to govern its own army of subordinated workers for free.

    Yet he wants to attract academics. LoL.

    Of course giving some virtual power to a group of editors could be attractive. Still, claiming that he couldn’t care less about academic standards as compared to internal policies (which he demands to accept on yes or no basis) speaks for itself.

  13. Philip Says:

    Hmm … even this happens.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: