Dusting off code: FederatedX

I have been meaning to, for months, get working on the FederatedX storage engine. There are a lot of great features it has -- such as better support for transactions as well as some architectural changes, thanks to the work of Antony Curtis. It now is designed so that the connection is abstracted into a federated_io_ class. This allows you to be able to subclass different connection schemes/drivers. For instance, I have the code for Federated ODBC that I need to take the code from and port to a federated_io_odbc class.

FederatedX has been pulled into an older MariaDB repository (thanks Antony!) that I need to merge with the latest, as well as figure out how to get the test suite to load the plugable storage engine. I've tried to add 'INSTALL PLUGIN federated SONAME 'ha_federatedx.so'' to the test, but the server expects ha_federated.so to be in a particular file location. That needs to be solved. I looked at PBXT's test suite but didn't see where they load the PBXT plugin: perhaps they compile it into the server vs. compiling as a plugable storage engine. So, as you can see, a table is created in MySQL 5.0 (stock install):

mysql> select version();
+------------------------+
| version() |
+------------------------+
| 5.0.51a-3ubuntu5.4-log |
+------------------------+
1 row in set (0.00 sec)

mysql> show create table users\G
*************************** 1. row ***************************
Table: users
Create Table: CREATE TABLE `users` (
`id` int(10) unsigned NOT NULL auto_increment,
`name` varchar(32) default NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=latin1


Then in Maria:

mysql> select version();
+--------------------+
| version() |
+--------------------+
| 5.1.32-maria-beta2 |
+--------------------+

mysql> INSTALL PLUGIN federated SONAME 'ha_federatedx.so'


mysql> CREATE TABLE `users` (
-> `id` int(10) unsigned NOT NULL auto_increment,
-> `name` varchar(32) default NULL,
-> PRIMARY KEY (`id`)
-> ) ENGINE=federated connection='mysql://fed:workonfederated_finally@192.168.1.118/test/users';
Query OK, 0 rows affected (0.04 sec)

mysql> select * from users;
Empty set (0.08 sec)

mysql> insert into users (name) values ('test user');
Query OK, 1 row affected (0.05 sec)

mysql> select * from users;
+----+-----------+
| id | name |
+----+-----------+
| 1 | test user |
+----+-----------+

Then back in MySQL:

mysql> select * from users;
+----+-----------+
| id | name |
+----+-----------+
| 1 | test user |
+----+-----------+

Great! I really love Federated, and have neglected it. Working at NorthScale, where open source projects are encouraged (And being done with the book!), I will be able to more actively develop these projects.

Goals

Some things I want to do with FederatedX:

1. Port to Drizzle!
2. Perhaps look at creating a Federated-only Drizzle, work in ODBC support. This could be used as a front-end to a number of remote, back-end databases. This would allow those using the LAMP stack to be able to develop their applications to work with LAMP while having the actual data stored on any type of RDBMS (or even a non-relational data source)
3. Like I said, ODBC support
4. Push-down conditions. One of the biggest issues with Federated (the concept even) is to try to get the remote data sources to do more work in producing a good result set, vs. retrieving a huge result set and making the Federated end do all the work. Also, simply reducing the network traffic and size of the result set.