Archive for category All

Slides from Developing Drizzle Replication Plugins Tutorial

Hi all!

So, Padraig, Toru, and myself teamed up yesterday at the MySQL Conference for about thirty or so attendees to discuss developing Drizzle plugins in C++. It was a set of slides that covered basic stuff all the way up through pretty advanced topics. We hope attendees got something out of it :)

Below are the slides from Padraig’s and my part of the tutorial which focused on plugin development basics and the replication plugin API in Drizzle. I’ve also tacked them onto my page of presentations.

Enjoy, and feel free to email me with comments and suggestions to SELECT REVERSE('moc.liamg@sepipyaj');

Developing Drizzle Replication Plugins


Open Office Impress slides
PDF slides


Topics included in the slides:

  • About the Drizzle Community and Expectations of Contributors
  • Getting started on Launchpad
  • Various features of Launchpad
  • Understanding the Source Code Directory Structure
  • Code walkthrough of Drizzle plugin basics
  • Drizzle’s System Architecture
  • Overview of Drizzle’s Replication System
  • Understanding Google Protobuffers
  • The Transaction message in Detail
  • In-depth code walkthrough of the Filtered Replicator module
  • In-depth code walkthrough of the Transaction Log module
  • Future of Drizzle replication – Publisher and Subscriber plugins

Is Anyone Else Looking Forward To…

…the new Clash of the Titans movie, coming out soon? I loved the original back in 1981. Can’t wait to see this one. :)

I would have linked to the new trailer site, but it was all Flash and took over two minutes to load even with broadband. I figured I’d save users the hassle…

O’Gara Cloud Computing Article Off Base

Maureen O’Gara, self-described as “the most read technology reporter for the past 20 years”, has written an article about Drizzle at Rackspace for one of Sys-con’s online zines called Cloud Computing Journal, of which she is an editor.

I tried commenting on Maureen’s article on their website, but the login system is apparently borked, at least for registered users who use OpenID, which it wants to still have a separate user ID and login. Note to sys-con.com: OpenID is designed so that users don’t have to remember yet another login for your website.

Besides having little patience for content-sparse websites that simply provide an online haven for dozens of Flash advertisements per web page, the article had some serious problems with it, not the least of which was using large chunks of my Happiness is a Warm Cloud article without citation. Very professional.

OK, to start with, let’s take this quote from the article:

Drizzle runs the risk of not being as stable as MySQL, because the Drizzle team is taking things out and putting other stuff in. Of course it may be successful in trying to create a product that’s more stable than MySQL. But creating a stable DBMS engine is something that has always taken years and years.

This is just about the most naïve explanation for whether a product will or will not be stable that I’ve ever read. If Maureen had bothered to email or call any one of the core Drizzle developers, they’d have been happy to tell her what is and is not stable about Drizzle, and why. Drizzle has not changed the underlying storage engines, so the InnoDB storage engine in Drizzle is the same plugin as available in MySQL (version 1.0.6).

The pieces of MySQL which were removed from Drizzle happen to be the parts of MySQL which have had the most stability issues — namely the additional features added to MySQL 5.0: stored procedures, views, triggers, stored functions, the INFORMATION_SCHEMA implementation, and server-side cursors and prepared statements. In addition to these removed features of MySQL, Drizzle also has no built-in Query Cache, does not support anything other than UTF-8 character sets, and has removed the MySQL replication system and binary logging — moving a rewrite of these pieces out into the plugin ecosystem.

The pieces that were added to Drizzle have mostly been done by adding plugins that provide functionality. Maureen, the reason this was done was precisely to allow for greater stability of the kernel by segregating new features and functionality into the plugin ecosystem, where they can be properly versioned and quarantined, therefore increasing kernel stability. It’s pretty much the biggest principle of Drizzle’s design…

The core developers of Drizzle (and much of the Drizzle community) would also have been happy to tell Maureen how the Drizzle team defines “stability”: when the community says Drizzle is stable — simple as that.

OK, so the next thing I took objection to is the following line:

Half of Rackspace’s customers are on MySQL so there’ll be some donkey-style nosing to get them to migrate.

I think my Rackspace colleagues might have quite a bit to say about the above. I haven’t seen any Rackers talking about mass migration from MySQL to Drizzle. As far as I have seen, the plan is to provide Drizzle as an additional service to Rackspace customers.

Rackspace evidently wants its new boys, who were not the core pillars of the MySQL engineering team, to hitch MySQL, er, Drizzle to Cassandra

MySQL != Drizzle. Implying that the two are equal do a disservice to both, as they have very different target markets and developer audiences.

The smart money is betting that even if a good number of high-volume web sites go down this route, an even higher number such as Facebook and Google will continue with relational databases, primarily MySQL.

Again, probably best to do your homework on this one, too. Facebook runs an amalgamation of a custom MySQL version and storage engines, distributed key-value stores, and Memcached servers. I would think that Facebook moving to Drizzle would be one tough migration. Thousands (tens of thousands?) of MySQL servers all running custom software and integrated into their caching layers is a huge barrier to entry, and not one I would expect a large site like Facebook to casually undertake. But, the same could be said about a move to SQL Server or Oracle, for that matter, and has little to do with Drizzle.

Google is moving away from using MySQL entirely. Mark Callaghan, previously at Google, has moved over to Facebook (possibly because of this trend at Google to get rid of MySQL), and Anthony Curtis, formerly of MySQL, then Google, left Google partially because of this reason.

OK, so the next quote got me really fired up because it demonstrates a complete lack of understanding (maybe not Maureen’s, but the unnamed source it’s from at least):

Somebody – sorry we forget who exactly – claimed that as GPL 2 code Drizzle “severely limits revenue opportunities. For Rackspace, the opportunity to have some key Drizzle developers on its payrolls basically comes down to a promotional benefit, trying to position Rackspace as particularly Drizzle-savvy in the eyes of the community and currying favor for its seemingly generous contributions. What’s unclear is whether they may develop some Drizzle-related functionality that they will then not release as open source and just rent out to Rackspace hosting customers…that would be a way for them to differentiate themselves from competitors and GPLv2 would in principle allow this.”

A few points to make about the above quote.

First, name your source. I find it difficult to believe that the most-read technology writer would not write down a source. Is it the same person you deliberately left out of a quote from my Happiness article? (why did you do that, btw?).

Second, the MySQL server source code is licensed under the GPL 2, and so is Drizzle’s kernel, because it is a derivative work of the MySQL server.

Let me be clear: Developers who contribute code to Drizzle do so under the GPLv2 if that contribution is in the Drizzle kernel. If the code contribution is a plugin, the contributor is free to pick whatever license they choose.

Third, licensing has little if anything to do with revenue at all. The license is besides the point. There are two things which dictate the company’s revenue derivation from software:

  1. Copyright ownership
  2. Principles of the Company

Drizzle, Rackspace, or any company a Drizzle contributor works for, does not have the copyright ownership of the MySQL source code, from which Drizzle’s kernel is derived. Oracle does. Therefore, companies do not have any right to re-sell Drizzle (under any license) without explicit permission from Oracle. Period. Has nothing to do with the GPLv2.

That said, contributors do have the right to make money on plugins built for the Drizzle server, and Rackspace, while not having expressed any interest to yours truly in doing so, has the right like any other Drizzle contributor, to make money on plugins its contributors create for Drizzle.

It is my knowledge (after actually having talked to Rackspace managers and decision makers), that Rackspace is not interested in getting into the business of selling commercial Drizzle plugins. Their core direction is to create value for their customers, and I fail to see how getting into the commercial software sales business meets that goal.

Next time, please feel free to contact myself or any other Drizzle contributor to get the low-down on Drizzle-related stuff. We’ll be nice. I promise.

jpipes.com is now joinfu.com

As you may have noticed, my blog has changed.  No more old and busted serendipity 0.9.  Hello Wordpress 2.9.  Yeah \0/

It was a manual data transformation I had to do in order to get all the posts and comments from my old blog to the new one, and I think I got everything transferred over correctly.  Please do let me know at REVERSE(‘moc.liamg@sepipyaj’) if you see anything out of the ordinary or something is messed up.  Thanks :)

Here’s to more blogging on joinfu.com ;)

An SQL Puzzle?

Dear Lazy Web,

What should the result of the SELECT be below? Assume InnoDB for all storage engines.

CREATE TABLE t1 (a int, b int);
INSERT INTO t1 VALUES (1,1),(1,2);
CREATE TEMPORARY TABLE t2 (a int, b int, PRIMARY KEY (a));
BEGIN;
INSERT INTO t2 VALUES (100,100);
CREATE TEMPORARY TABLE IF NOT EXISTS t2 (PRIMARY KEY (a)) SELECT * FROM t1;
 
# The above statement will correctly produce an ERROR 23000: Duplicate entry '1' for key 'PRIMARY'
# What should the below result be?
 
SELECT * FROM t2;
COMMIT;

Funny Bush Joke

Friend sent me this one; thought it was worthy of an entry… :)

Q:
How many Bush Administration officials does it take to screw in a light bulb?

A:
None. There is nothing wrong with the light bulb; its conditions are improving every day. Any reports of its lack of incandescence are a delusional spin from the liberal media. That light bulb has served honorably, and anything you say undermines the lighting effect. Why do you hate freedom?

Ha. Truly democratic enlightenment. No pun intended.

The SEO Myth

I’ve gotten more and more requests recently to help clients with search engine optimization. Most of the time, the request comes in like “hey, can you help XYZ corp get a lot of hits on their new site?”. For the most part, folks seem to have this impression that you can follow some strict mantra of “SEO” rules and tricks and you’ll somehow get a massive stream of visitors to your site.

I spend a lot of time trying to dispel this myth for my clients.

It’s not that I’m that pessimistic. I mean, I am, but not monstrously so. I just want people to have a realistic idea of what really makes a site, or for that matter a particular product, attract an audience…

Welcome Home from London

As if getting stranded in Philadelphia while US Airways sorted out about 1800 passengers from California wasn’t bad enough, I had three voicemails on my phone when I returned from neighbours talking about a tree down at my house. Columbus got smacked with a tremendous ice storm that knocked down branches and took out power to hundreds of thousands while I was in England, happily retreating from work and clients…

Good times. Well, at least the tree didn’t hit the house; it only took out the lamp-post, which I wasn’t entirely fond of in any case.