Win a Server for Life (some restrictions apply)
————————————————
The riseup birds have big big dreams! But, alas, we are kept super busy maintaining and improving the current services we provide. One thing we are good at is building infrastructure (and acquiring donated hardware).

So good, in fact, that we want to give some of it away in a contest we are calling the “Win a Server for Life (some restrictions apply) 2008 Extravaganza”.

We got the servers, you got the talent, baby! Here’s the deal: We provide your collective with free hardware, bandwidth, and rackspace for the life of the server. You provide an important service to a radical social movement or local community. So long as you are using the capacity of the server, we will maintain it and host it for you.

We are particularly keen on ideas to create new services that do not currently exist, services that are difficult to find, or services that meet a need for an underserved community. Examples might include VoIP for radicals, rails hosting that keeps no logs, an encrypted backup service for activists, secure DNS services, and so on.

The server in question is a quad core 2.8ghz xeon with 4gb of ram and three disk bays. Don’t bother with a formal proposal. Drop us a note at collective@riseup.net with your idea and what you are about. We will talk, figure out who we know in common, and make sweet data together.

I used deadwood to generate the menus system for me- mostly worked, but I’ve still had to do some cleanup, probably because my own menus weren’t that well designed. I tried having deadwood convert the full module, but kept running into errors. I think it has a greater difficulty with complex modules as would be expected. At anyrate, there is a functional patch for Drupal-5 to upgrade to six here: http://drupal.org/node/269126

update: recovery process has started…. I’ve lost all the comments, but at least I’m finding some of the content

Drupal employs the Throttle module to handle high load situations. It looks at traffic patterns on the site to determine if a throttle condition should be employed. In considering how to limit the activity of Media Mover on a server this is an obvious consideration. One of the things that makes Media Mover (for most use cases) different than the needs of modules effected by Throttle is that it is not bound by the traffic occuring on the site- it may well be a factor, but it is not the only consideration. If a Media Mover configuration is using FFmpeg to transcode files, it is almost exclusively CPU bound. Of course there is the Apache process that kicks off cron, some degree of Drupal processing, and perhaps a MySQL hit or two, but largely FFmpeg wants to make use of the server CPU. Media Mover will cycle through sets of files that it is loading from MySQL, but again, in terms of its total impact, mostly we are looking at FFmpeg hitting the CPU.

In this case, if we want to limit Media Mover from running when a server is starting to get overloaded on its CPU we need to check the CPU load. I’ve written a fairly basic utility that does this on Linux / Unix based servers on Media Mover’s cron processing script. It allows other scripts to still access the API without being throttled, but Cron is stopped if load creeps beyond what you set it for.

You can set the threshold where Media Mover should stop, and select which average you want to use as the threshold- the one, five or 15 minute load averages. Once the load drops below the threshold, the Media Mover Cron runs will start up again.

Poking around, I didn’t find much information on how to actually write your own Panels integration, so after a few fits and starts, I just started from block.inc as an example and then have abstracted it here.
I wanted to do this in a fairly simple way- a basic definition function and then a set of functions that pass these definitions off to theme functions. I’m making an example out of my code for people to use- hopefully it is useful to see how to set this up.

First we’re going to define what the items are that will be used as panes in Panels. This is a custom function, not a Panels function.
/**
* Defines the panel content for this module.
* @return array
* array has a key which will be used as a function call and a title
*/
function mymod_panels_content_define() {
$items = array(
'mymod_item_1' =>; t('This is pane one'),
'mymod_item_2' =>; t('This is pane two'),
);
return $items;
}
Now we need to declare the theme functions that will be used. Obviously these are rather simple for demonstration purposes, but you get the idea. Note that you would name these in relationship to your declarations above. Here is the first pane:
/**
* display the content for item 1
* @param array $conf
* @return string
* html return
*/
function theme_mymod_item_1($conf) {
return "my module item 1 theme function output";
}
And the second- again, here is where you would add your custom functions.
/**
* display the content for item 2
* @param array $conf
* @return string
* html return
*/
function theme_mymod_item_2($conf) {
return "my module item 2 theme function output";
}
Now we need to tell Panels about our code. This declares what functions are responsible for handling the Panels callbacks. These are Panels functions.
/**
* Callback function to supply a list of content types.
* @return array
*/
function mymod_panels_panels_content_types() {
$items['mymod panes'] = array(
'title' =>; t('My Module panes'),
'content_types' =>; 'mymod_panels_admin_content_types',
'title callback' =>; 'mymod_panels_admin_title',
'add callback' =>; 'mymod_panels_admin_add',
'render callback' =>; 'mymod_panels_content',
);
return $items;
}
Now that Panels knows about our code, we tell it how to handle the content types. We’re going to get the items that we declared first and then assign them a few additional pieces of information to make them appear in the right places in the admin interface.
/**
* Return all block content types available.
* @return array
*/
function mymod_panels_admin_content_types() {
$types = array();
foreach (mymod_panels_content_define() as $id =>; $title) {
$info = array(
'title' =>; $title,
'category' =>; array(t('My Module panes')),
'icon' =>; drupal_get_path('module', 'mymod_panels') .'/images/mymod.gif',
'id' =>; $id,
);
$types["mymod_panels-$id"] = $info;
}
return $types;
}
When an admin adds our pane, we need to add some additional information to the pane so that we can access it during the rendering phase. In the above function, we keyed the array data with the module and function data so we can pull it out easily. Now we just need to put that data into a form:
/**
* Returns the form for a new block.
*/
function mymod_panels_admin_add($id, $parents, $conf = array()) {
list($conf['module'], $conf['delta']) = explode('-', $id, 2);
$form['module'] = array(
'#type' =>; 'value',

'#value' =>; $conf['module'],
);
$form['delta'] = array(
'#type' =>; 'value',
'#value' =>; $conf['delta'],
);
return $form;
}
Now that we’ve added this data into the form, it is passed through to the render phase. In this example, to render the pane, we are doing something simple- just pass the call to a coresponding theme function- one of the ones we declared above.
/**
* call the content to generate for the pane in question
* calls a theme(module_name_pane_name, $conf);
* @param array $conf
* @return object
*/
function mymod_panels_content($conf) {
$function = $conf['module'] .'_'. $conf['delta'];
$block = new stdClass();
$block->;content = theme("$function", $conf);
return $block;
}
Finally, we just need a function that declares the title of the pane to Panels:
/**
* Returns the administrative title for a type.
* @param array $conf
*/
function mymod_panels_admin_title($conf) {
$panels = mymod_panels_content_define();
return $panels[$conf['delta']];
}
Now you’ve got a fast way to integrate your custom content into Panels. There might be easier/faster ways to do this, but I haven’t found an examples to build from. You can download the example file here as well: mymod_panels.php

The problem is that Media Mover is more of an API than it is an interface. In fact, you’d be hard pressed to really call it an interface if you’re an end user. A while back, I released the Auto Run module for Media Mover which runs Media Mover configurations when a node is created or updated. This really set the groundwork for starting to move toward running Media Mover configurations on specific user interactions- the ones that I think a majority of people are looking for. This approach- the transcode on create/update content- while workable, still does not give the end user control of how page layout works. For some, this is a fine middle ground approach, for others, unworkable. From my perspective, Asset provides a solid way for users to add content to their nodes. Of course WYSISYG editors are problematic, but they are not going anywhere, and hopefully, their HTML will just get better. Asset goes along way toward improving the integration between WYSIWYG editors and Drupal. Media Mover can remain a work horse for rich media, but it can also play nice when it comes to offering up its services.

From what I’ve heard from other Drupal admins and people who use the web in general, they do not want to be bothered by file formats, complex preferences, or sorting out what formats are playable and what you need for the non-playable ones. Asset, Media Mover, and FLVMediaPlayer combined offer a step forward in addressing this- users can upload “video” and get “video” on the web, without a tremendous amount of knowledge of what is actually going on. This gives the admin the opportunity to fine tune what options are available, while users get to have the fun of having a full transcoding system at their finger tips. Hence a union between Asset and Media Mover seems a perfect thing.

I did a little screen cast to show how this works. It shows the process of enabling the modules, making sure that there are profiles in the FLVMediaPlayer, setting up Asset, and the process of actually uploading and inserting with Asset. If you are not familiar with these modules, should still make some sense, though it will probably be more meaningful if you are familiar with Asset.

This screen shot shows the listing of the Media Mover configurations that are available to run. Note that the administrator should choose configurations that are harvesting from nodes- otherwise it is just wasted cycles.

Once the 1.0 release is stabilized for both, I’ll write some functions that allow modules to define FLV Media Player profiles and Media Mover configurations in custom modules so that these can be kept under version control, making it easier to deploy changes via version control.

I just realized that my talk on Media Mover at the Seattle Drupal Users’ Group from this past June was posted on Blip. I guess I’m a bit behind the times! This is similar to the talk that I did in NYC with less command line and more talking. Interestingly, some of the items that are “future plans” have actually been completed already and are in the alpha releases of Media Mover 1.

The Seattle group is really impressive- super well organized, really well attended weekend with great talks and great people. Both Robin and Gregory really knocked this one out of the park!

I’ve setup FFmpeg wrapper to use configuration files. For each output format, a format file can be create which defines what kinds of options are available (codecs, bit rates, sample rates, etc). These files are loaded via ajax and any form that conforms to a set of rules can be updated on the fly as the user selects different options.

While this currently only works for Media Mover’s FFmpeg processing module, it would work for any module that wanted to implement the setup. Further, people who have experience with specific kinds of output formats can contribute back configuration files that help improve the output from FFmpeg.

The screencast below shows this process in action- I’m adding a new configuration to Media Mover and then selecting different output options for FFmpeg. While I only have one configuration complete at this time (FLV) switching between other options grabs the default (ie: all) options. It also turns the advanced configurations off when you select an option that uses the default- this way FFmpeg will take it’s best guess at converting unless you specifically tell it to.

On top of this, I’ve added support in mm_ffmpeg to enter in your own command- while this is potentially dangerous, it gives people with advanced needs of FFmpeg the ability to run what they need.

]]> http://www.24b6.net/2008/07/28/ffmpeg-wrapper-auto-configuration/feed/ http://www.24b6.net/2008/07/09/simpletest/ http://www.24b6.net/2008/07/09/simpletest/#comments Wed, 09 Jul 2008 14:29:41 +0000 arthur http://www.24b6.net/?p=227 Doug Green just did a great presentation for us on SimpleTest. It is really inspiring to see how straight forwarding adding testing is to modules. I spent a few minutes to add the frame work to Media Mover and very very… well, simple test. This first one is not much but for the 1.0 release, I will start adding more…

Two modules have been incorporated into Media Mover- mm_token and mm_workflow_ng to do this integration. Tokens are provided from the mm_token module and the events are defined in mm_workflow_ng.

Currently things are in CVS, and you can track the issue there….

Here you can see the options that the mm_dir module gives on a configuration screen:

The code is in CVS and will be released with the next Media Mover release.

See the discussion here: http://drupal.org/node/258956#comment-900776
UPDATE: now also supports storing to the filesystem as well.

Media Mover presents similar challenges for websites.You want to try out major changes before they go live- often on a different server or at least a different environment from your live one. You don’t want to bog down your server with extra processing until you know for sure you’ve got things right. Further, it gives you a chance to test out different configurations of processing to tweak the final output for your ends.

The problem however, is moving those configurations from one environment to another. It’s not that hard to do this by hand, but there is room for error, and doing lots of this can get tiresome very quickly.

I sat down this morning and started shelling out the import / export system for Media Mover. At the moment, it’s quite simple and doesn’t allow the defining of configurations in other modules, however, that should be fairly easy to implement. What it does do is straight forward exporting of an existing configuration and importing of the same configuration. The only real error checking that it does is to make sure that the same models are enabled on the import- this prevents some potentially nasty things from happening since there isn’t a whole lot of error checking once Media Mover starts processing.

The GUI is similar to views- clicking on the main export button gives you the options of which configurations to output, there is also a single link from the configuration landing page which directly exports that configuration.

That’s the extent of the export functionality. Once the user has the code, they can paste this code into the import textarea:

The code is basically the configuration object re-rendered in a textarea. Again, very similar to the approach taken by views:

In this final image, you can see the new view. I renamed it in the code so that it would have a new name on import just to make it obvious- by default this doesn’t happen, it’s just for demonstration purposes. If you’re interested in checking it out, take a peek at the DRUPAL-5 CVS branch. All the code is there.