PulseConf 2012: Report

For those of you who missed my previous updates, we recently organised a PulseAudio miniconference in Copenhagen, Denmark last week. The organisation of all this was spearheaded by ALSA and PulseAudio hacker, David Henningsson. The good folks organising the Ubuntu Developer Summit / Linaro Connect were kind enough to allow us to colocate this event. A big thanks to both of them for making this possible!

The conference was attended by the four current active PulseAudio developers: Colin Guthrie, Tanu Kaskinen, David Henningsson, and myself. We were joined by long-time contributors Janos Kovacs and Jaska Uimonen from Intel, Luke Yelavich, Conor Curran and Michał Sawicz.

We started the conference at around 9:30 am on November 2nd, and actually managed to keep to the final schedule(!), so I’m going to break this report down into sub-topics for each item which will hopefully make for easier reading than an essay. I’ve also put up some photos from the conference on the Google+ event.

Mission and Vision

We started off with a broad topic — what each of our personal visions/goals for the project are. Interestingly, two main themes emerged: having the most seamless desktop user experience possible, and making sure we are well-suited to the embedded world.

Most of us expressed interest in making sure that users of various desktops had a smooth, hassle-free audio experience. In the ideal case, they would never need to find out what PulseAudio is!

Orthogonally, a number of us are also very interested in making PulseAudio a strong contender in the embedded space (mobile phones, tablets, set top boxes, cars, and so forth). While we already find PulseAudio being used in some of these, there are areas where we can do better (more in later topics).

There was some reservation expressed about other, less-used features such as network playback being ignored because of this focus. The conclusion after some discussion was that this would not be the case, as a number of embedded use-cases do make use of these and other “fringe” features.

Increasing patch bandwidth

Contributors to PulseAudio will be aware that our patch queue has been growing for the last few months due to lack of developer time. We discussed several ways to deal with this problem, the most promising of which was a periodic triage meeting.

We will be setting up a rotating schedule where each of us will organise a meeting every 2 weeks (the period might change as we implement things) where we can go over outstanding patches and hopefully clear backlog. Colin has agreed to set up the first of these.

Routing infrastructure

Next on the agenda was a presentation by Janos Kovacs about the work they’ve been doing at Intel with enhancing the PulseAudio’s routing infrastructure. These are being built from the perspective of IVI systems (i.e., cars) which typically have fairly complex use cases involving multiple concurrent devices and users. The slides for the talk will be put up here shortly (edit: slides are now available).

The talk was mingled with a Q&A type discussion with Janos and Jaska. The first item of discussion was consolidating Colin’s priority-based routing ideas into the proposed infrastructure. The broad thinking was that the ideas were broadly compatible and should be implementable in the new model.

There was also some discussion on merging the module-combine-sink functionality into PulseAudio’s core, in order to make 1:N routing easier. Some alternatives using te module-filter-* were proposed. Further discussion will likely be required before this is resolved.

The next steps for this work are for Jaska and Janos to break up the code into smaller logical bits so that we can start to review the concepts and code in detail and work towards eventually merging as much as makes sense upstream.

Low latency

This session was taken up against the background of improving latency for games on the desktop (although it does have other applications). The indicated required latency for games was given as 16 ms (corresponding to a frame rate of 60 fps). A number of ideas to deal with the problem were brought up.

Firstly, it was suggested that the maxlength buffer attribute when setting up streams could be used to signal a hard limit on stream latency — the client signals that it will prefer an underrun, over a latency above maxlength.

Another long-standing item was to investigate the cause of underruns as we lower latency on the stream — David has already begun taking this up on the LKML.

Finally, another long-standing issue is the buffer attribute adjustment done during stream setup. This is not very well-suited to low-latency applications. David and I will be looking at this in coming days.

Merging per-user and system modes

Tanu led the topic of finding a way to deal with use-cases such as mpd or multi-user systems, where access to the PulseAudio daemon of the active user by another user might be desired. Multiple suggestions were put forward, though a definite conclusion was not reached, as further thought is required.

Tanu’s suggestion was a split between a per-user daemon to manage tasks such as per-user configuration, and a system-wide daemon to manage the actual audio resources. The rationale being that the hardware itself is a common resource and could be handled by a non-user-specific daemon instance. This approach has the advantage of having a single entity in charge of the hardware, which keeps a part of the implementation simpler. The disadvantage is that we will either sacrifice security (arbitrary users can “eavesdrop” using the machine’s mic), or security infrastructure will need to be added to decide what users are allowed what access.

I suggested that since these are broadly fringe use-cases, we should document how users can configure the system by hand for these purposes, the crux of the argument being that our architecture should be dictated by the main use-cases, and not the ancillary ones. The disadvantage of this approach is, of course, that configuration is harder for the minority that wishes multi-user access to the hardware.

Colin suggested a mechanism for users to be able to request access from an “active” PulseAudio daemon, which could trigger approval by the corresponding “active” user. The communication mechanism could be the D-Bus system bus between user daemons, and Ștefan Săftescu’s Google Summer of Code work to allow desktop notifications to be triggered from PulseAudio could be used to get to request authorisation.

David suggested that we could use the per-user/system-wide split, modified somewhat to introduce the concept of a “system-wide” card. This would be a device that is configured as being available to the whole system, and thus explicitly marked as not having any privacy guarantees.

In both the above cases, discussion continued about deciding how the access control would be handled, and this remains open.

We will be continuing to look at this problem until consensus emerges.

Improving (laptop) surround sound

The next topic dealt with being able to deal with laptops with a built-in 2.1 channel set up. The background of this is that there are a number of laptops with stereo speakers and a subwoofer. These are usually used as stereo devices with the subwoofer implicitly being fed data by the audio controller in some hardware-dependent way.

The possibility of exposing this hardware more accurately was discussed. Some investigation is required to see how things are currently exposed for various hardware (my MacBook Pro exposes the subwoofer as a surround control, for example). We need to deal with correctly exposing the hardware at the ALSA layer, and then using that correctly in PulseAudio profiles.

This led to a discussion of how we could handle profiles for these. Ideally, we would have a stereo profile with the hardware dealing with upmixing, and a 2.1 profile that would be automatically triggered when a stream with an LFE channel was presented. This is a general problem while dealing with surround output on HDMI as well, and needs further thought as it complicates routing.

Testing

I gave a rousing speech about writing more tests using some of the new improvements to our testing framework. Much cheering and acknowledgement ensued.

Ed.: some literary liberties might have been taken in this section

Unified cross-distribution ALSA configuration

I missed a large part of this unfortunately, but the crux if the discussion was around unifying cross-distribution sound configuration for those who wish to disable PulseAudio.

Base volumes

The next topic we took up was base volumes, and whether they are useful to most end users. For those unfamiliar with the concept, we sometimes see sinks/sources where which support volume controls going to > 0dB (which is the no=attenuation point). We provide the maximum allowed gain in ALSA as the maximum volume, and suggest that UIs show a marker for the base volume.

It was felt that this concept was irrelevant, and probably confusing to most end users, and that we suggest that UIs do not show this information any more.

Relatedly, it was decided that having a per-port maximum volume configuration would be useful, so as to allow users to deal with hardware where the output might get too loud.

Devices with dynamic capabilities (HDMI)

Our next topic of discussion was finding a way to deal with devices such as those HDMI ports where the capabilities of the device could change at run time (for example, when you plug out a monitor and plug in a home theater receiver).

A few ideas to deal with this were discussed, and the best one seemed to be David’s proposal to always have a separate card for each HDMI device. The addition of dynamic profiles could then be exploited to only make profiles available when an actual device is plugged in (and conversely removed when the device is plugged out).

Splitting of configuration

It was suggested that we could split our current configuration files into three categories: core, policy and hardware adaptation. This was met with approval all-around, and the pre-existing ability to read configuration from subdirectories could be reused.

Another feature that was desired was the ability to ship multiple configurations for different hardware adaptations with a single package and have the correct one selected based on the hardware being run on. We did not know of a standard, architecture-independent way to determine hardware adaptation, so it was felt that the first step toward solving this problem would be to find or create such a mechanism. This could either then be used to set up configuration correctly in early boot, or by PulseAudio for do runtime configuration selection.

Relatedly, moving all distributed configuration to /usr/share/..., with overrides in /etc/pulse/... and $HOME were suggested.

Better drain/underrun reporting

David volunteered to implement a per-sink-input timer for accurately determining when drain was completed, rather than waiting for the period of the entire buffer as we currently do. Unsurprisingly, no objections were raised to this solution to the long-standing issue.

In a similar vein, redefining the underflow event to mean a real device underflow (rather than the client-side buffer running empty) was suggested. After some discussion, we agreed that a separate event for device underruns would likely be better.

Beer

We called it a day at this point and dispersed beer-wards.

User experience

David very kindly invited us to spend a day after the conference hacking at his house in Lund, Sweden, just a short hop away from Copenhagen. We spent a short while in the morning talking about one last item on the agenda — helping to build a more seamless user experience. The idea was to figure out some tools to help users with problems quickly converge on what problem they might be facing (or help developers do the same). We looked at the Ubuntu apport audio debugging tool that David has written, and will try to adopt it for more general use across distributions.

Hacking

The rest of the day was spent in more discussions on topics from the previous day, poring over code for some specific problems, and rolling out the first release candidate for the upcoming 3.0 release.

And cut!

I am very happy that this conference happened, and am looking forward to being able to do it again next year. As you can see from the length of this post, there are lot of things happening in this part of the stack, and lots more yet to come. It was excellent meeting all the fellow PulseAudio hackers, and my thanks to all of them for making it.

Finally, I wouldn’t be sitting here writing this report without support from Collabora, who sponsored my travel to the conference, so it’s fitting that I end this with a shout-out to them. :)

November 06, 2012 11:04 AM

grsec and PulseAudio (and Gentoo)

This problem seems to bite some of our hardened users a couple of times a year, so thought I’d blog about it. If you are using grsec and PulseAudio, you must not enable CONFIG_GRKERNSEC_SYSFS_RESTRICT in your kernel, else autodetection of your cards will fail.

PulseAudio’s module-udev-detect needs to access /sys to discover what cards are available on the system, and that kernel option disallows this for anyone but root.

October 30, 2012 08:49 AM

PulseConf Schedule

David has now published a tentative schedule for the PulseAudio Mini-conference (I’m just going to call it PulseConf — so much easier on the tongue).

For the lazy, these are some of the topics we’ll be covering:

• Vision and mission — where we are and where we want to be
• Improving our patch review process
• Routing infrastructure
• Improving low latency behaviour
• Revisiting system- and user-modes
• Devices with dynamic capabilities
• Improving surround sound behaviour
• Separating configuration for hardware adaptation
• Better drain/underrun reporting behaviour

Phew — and there are more topics that we probably will not have time to deal with!

For those of you who cannot attend, the Linaro Connect folks (who are graciously hosting us) are planning on running Google+ Hangouts for their sessions. Hopefully we should be able to do the same for our proceedings. Watch this space for details!

p.s.: A big thank you to my employer Collabora for sponsoring my travel to the conference.

October 29, 2012 12:45 PM

PulseConf!

For those of you who missed it, your friendly neighbourhood PulseAudio hackers are converging on Copenhagen in a month to discuss, plan and hack on the future of PulseAudio.

We’re doing this for the first time, so I’m super-excited! David has posted details so if this is of interest to you, you should definitely join us!

October 04, 2012 08:41 AM

Pulseaudio conference – less than a month away!

The first PulseAudio conference is approaching quickly. This is a shoutout for people who might be interested, but missed the mailinglist announcement.

The conference will be Friday 2nd November 2012, and colocated with Ubuntu Developer Summit and Linaro Connect, in Bella Center, Copenhagen.

There is no attendance fee, but you’ll need a UDS or LC registration (which are also free) to be able to get into the conference area. (Note: LC might be the safer bet here as UDS ends on Thursday)

There had been a few topics brought up on the mailinglist, but we welcome anyone who would like to contribute to constructive discussions and help to shape the future of PulseAudio!

If you would like to attend, how about you write an email to the pulseaudio-discuss mailinglist with the topic(s) you want to bring up, and maybe a small presentation with your background and interests? Of course, if you mostly want to listen in rather than bring your own topics, that’s okay too!

October 04, 2012 08:32 AM

Top five wrong ways to fix your audio

The audio stack in Linux/Ubuntu evolves over time. What used to be good advice is not necessarily good advice anymore. (That also means, that if you happen to read this blog post in 2019 or something, don’t trust it!)
Here are some things that people try, and sometimes they even fix the problem, but are often bad in one way or the other. Or at least, they have side effects one needs to be aware of. So – while there are valid exceptions, as a rule of thumb, don’t do the following:

5. Don’t add your user to the “audio” group

A user has access to the audio card if that person is either logged in – both VT and GUI login counts, but not SSH logins, or if that user is in the “audio” group. However, on the level of access we’re talking about here, only one user has access at a time. So the typical problem scenario goes like:

• User Homer has an audio issue, and tries to fix it by adding himself to the audio group. This doesn’t help to resolve the problem.
• Homer discovers his audio is muted, and unmutes it. Happy to have his audio issue resolved, he forgets he’s still in the audio group, or doesn’t realise it leads to problems.
• User Marge comes and wants to borrow the computer. Homer does a fast-user-switching so Marge can log in.
• Because Homer is in the audio group, he has still access to the audio device. If some software, e g PulseAudio, has the audio device opened, it blocks access to other software trying to use it.
• Now Marge has an audio issue!

I’ve written a longer article about the audio group here. In short, there are some usages for it, including that it is also the standard group name for assigning realtime priorities when used together with JACK. But don’t leave a user in the audio group unless you have a good reason.

4. Don’t try different “model” strings

A common way to try to get HDA Intel soundcards to work is to edit /etc/modprobe.d/alsa-base.conf and add the following line:

options snd-hda-intel model=[something]

…where [something] are values you find in some file. Contrary to official documentation, this is in most cases obsolete. In particular, avoid model=generic – that is almost guaranteed to give you trouble. In many cases, when trying different models, you will find that you might fix one thing but break another.
In fact, there is only one model to try, and that is model=auto. If your machine happen to be one of those quirked to use an older model parser, changing to model=auto can improve the situation.

Instead do:
It still happens that BIOS/UEFI assigns the wrong values to pin nodes, which causes an output or input not to work correctly. If so, I recommend trying to tweak this with hda-jack-retask.
In some cases, trying different modules can actually be okay – sometimes, these models point to lightweight fixups instead of the earlier, more heavyweight code that was used in previous kernels. (In this context, I have to mention that Takashi Iwai has done a fantastic job of converting the older models to the newer auto-parser.)

3. Don’t upgrade ALSA drivers by following random blog posts

I’ve seen far too many people reporting bugs on Launchpad where they’ve been following some random blog post that tells you how to upgrade ALSA, and are having audio issues as a result. These guides are of varying quality and often come without good uninstall instructions, so you have no way to revert in case the upgrade did not solve your problem, or broke something else.

First, something not everybody is aware of: 95% of ALSA code is in the kernel, and follows the kernel’s release cycle. That means that even if “/proc/asound/version” says something that was released a year or two ago, don’t panic. It’s the kernel release that tells you how new your sound drivers are, so if you have a new kernel, and you see an ALSA release coming out, you are unlikely to gain from an upgrade.

Instead do:
In some case you do have an old kernel, and newer sound drivers can be worth a try. The Ubuntu Audio Developer’s team provides daily snapshot drivers for HDA Intel cards. Guide is available here and it also comes with proper uninstall instructions.
In the past we have also provided drivers for other cards, but due to the maintenance required to keep this up-to-date, in combination with that the vast majority of people’s bugs concern HDA Intel anyway, this support has been discontinued.

2. Don’t purge PulseAudio

First, PulseAudio itself isn’t perfect, some of the bindings to PulseAudio aren’t perfect, and some of the drivers are not perfect in the way PulseAudio wants to use it either. So there might be valid reasons to temporarily move it out of your way, even if it would be better to actually fix the problem and submit a bug fix patch (if you’re capable of doing so).
But don’t try uninstalling the PulseAudio package, as it has far too many dependencies.

Instead do:
If you just need direct access to your sound card, you can run the “pasuspender” command. You can either run “pasuspender” (in a terminal) to make PulseAudio stay away for the duration of the application. Or if you think that’s simpler, just run “pasuspender bash” (in a terminal), start your application through the menu/dash/whatever you prefer, and when you’re done, write “exit” in the terminal.
If you need to stop the PulseAudio process completely, execute these commands:

echo autospawn=no > ~/.pulse/client.conf
pulseaudio -k

If you need PulseAudio back again, remove ~/.pulse/client.conf, then try to start an application that uses PulseAudio, and it should start automatically.

Unexpected side effects:

• The Gnome sound settings, the sound indicator and the volume up/down keys relies on PulseAudio, so they won’t work when PulseAudio is off.
• PulseAudio mixes audio, so that means that only one application at a time can output audio if PulseAudio is disabled (and you aren’t using some other sound server).
• Several applications have PulseAudio backends. Some of them will need reconfiguration to use ALSA directly, some will just automatically redirect themselves, and some won’t work at all.
• Bluetooth audio might not work without PulseAudio.

1. Don’t replace ALSA with OSS

OSS was the standard used before ALSA came along. These days, ALSA is much better, both when it comes to hardware support, and when it comes to how much software that supports outputting sound to either sound system. OSS is also entirely unsupported, at least by Ubuntu. In addition, I’m not sure exactly how to get back to a working system after you’ve tried OSS…!

Instead do:
If you know your problem is in ALSA, either drivers or userspace, try to track down and/or fix the bug, and talk to us about it. If you’re running Ubuntu, file a bug against the alsa-driver package. You can also contact the alsa-devel mailinglist. While we won’t guarantee responses due to the high volume of bugs/traffic, we are often able to help out.

Final notes

Note 1. HDA Intel cards are the built-in audio inputs and outputs on your motherboard (at least if you bought your computer after ~2006 or so). HDMI and DisplayPort audio are also HDA Intel cards, but they are covered in more detail here.

Note 2. I have had some problems with spammers posting spam comments to my blog post. I don’t want to spend too much time just reading spam and marking it as such, so I might close for comments in a relatively short period. Sorry for the inconvenience.

July 13, 2012 03:49 PM

Writing a simple PulseAudio module.

This post will describe the (little) progress I've made in the first week of coding and will try to provide a short but clear introduction to writing PulseAudio modules. A similar article exists on the PulseAudio wiki page, but I shall post this nevertheless as the next articles will be based on the code presented here. Finally, before checking the wiki page or going through this article, it is useful to know how to compile and run PulseAudio from git. Note that, as the README file says, the configure script must be run with the CFLAGS="-ggdb3 -O0" LDFLAGS="-ggdb3" environment variables, to be able to start from the build directory.

PulseAudio modules are required to have two functions: pa__init(pa_module*m) and pa__done(pa_module*m), which are called when the module is loaded and, respectively, unloaded (either by PulseAudio or because it has finished whatever it was doing). Other than  these functions, PulseAudio recognises a few others that provide some information about the module (such as the author or the description). However, these functions are not generally implemented directly, but rather created through the following macros defined in pulsecore/module.h:

PA_MODULE_AUTHOR(s)
PA_MODULE_DESCRIPTION(s)
PA_MODULE_USAGE(s)
PA_MODULE_VERSION(s)
PA_MODULE_DEPRECATED(s)
PA_MODULE_LOAD_ONCE(b)

pulsecore/module.h also contains the pa_module struct definition, which is explained on the Module API wiki page. One particularly relevant field of the structure is the userdata field, because it's the only way to carry information from pa__init to pa__done. This is why modules will generally define a userdata struct that contains all the relevant module data, and that is used by pa__done to free whatever memory is allocated in pa__init. This struct is also used to pass data to core hooks that handle various events.

Another important field in the pa_module structure is the core field. This contains a pa_core struct, which is defined in pulsecore/core.h and is described on the Core API wiki page. This field is particularly relevant because it is used when creating hooks.

Creating and freeing hook slots is done through two functions defined in pulsecore/hook-list.h:

pa_hook_slot* pa_hook_connect(pa_hook *hook, pa_hook_priority_t prio, pa_hook_cb_t cb, void *data);
void pa_hook_slot_free(pa_hook_slot *slot);

The core hook types are stored in the hooks array in pa_core, which is indexed according to the pa_core_hook enum defined in pulsecore/core.h. As an example, the hook that is called after a card has been created is referred to by m->core->hooks[PA_CORE_HOOK_CARD_PUT].

pa_hook_priority_t is another enum defined in pulsecore/hook-list.h and can currently take the values PA_HOOK_EARLY, PA_HOOK_NORMAL or PA_HOOK_LATE. However, these are just integers and the call can be adjusted to control the order in which hooks are run (e.g. PA_HOOK_LATE+10 will run after PA_HOOK_LATE).

The pa_hook_cb_t is a function pointer defined in pulsecore/hook-list.h, with the following definition:

typedef pa_hook_result_t (*pa_hook_cb_t)(
        void *hook_data,
        void *call_data,
        void *slot_data);

and the last parameter is used for whatever user data needs to be passed from the module to the hook callback function.

Putting all this together, the following module is able to detect and log whenever a new card is plugged in:

#ifdef HAVE_CONFIG_H
#include <config.h>
#endif

#include <pulse/proplist.h>
#include <pulse/xmalloc.h>

#include <pulsecore/card.h>
#include <pulsecore/core.h>
#include <pulsecore/log.h>
#include <pulsecore/module.h>

#include "module-desktop-notifications-symdef.h"

PA_MODULE_AUTHOR("Ștefan Săftescu");
PA_MODULE_DESCRIPTION("Integration with the Desktop Notifications specification.");
PA_MODULE_VERSION(PACKAGE_VERSION);
PA_MODULE_LOAD_ONCE(TRUE);

struct userdata {
    pa_hook_slot *card_put_slot;
};


static pa_hook_result_t card_put_cb(pa_core *c, pa_card *card, void* userdata) {
    pa_log_info("Card detected: %s.", pa_proplist_gets(card->proplist, PA_PROP_DEVICE_DESCRIPTION));
    return PA_HOOK_OK;
}


int pa__init(pa_module*m) {
    struct userdata *u;
   
    m->userdata = u = pa_xnew(struct userdata, 1);
   
    u->card_put_slot = pa_hook_connect(&m->core->hooks[PA_CORE_HOOK_CARD_PUT], PA_HOOK_LATE, (pa_hook_cb_t) card_put_cb, u);
   
    return 0;
}

void pa__done(pa_module*m) {
    struct userdata *u;
    pa_assert(m);

    if (!(u = m->userdata))
        return;

    if (u->card_put_slot)
        pa_hook_slot_free(u->card_put_slot);

    pa_xfree(u);
}

June 15, 2012 02:07 AM

Welcome

Welcome to my blog!

I've started this blog to document the progress on my Google Summer of Code 2012 project for PulseAudio, but I will hopefully keep posting about other (more or less) interesting things I'm working on.

The project aims to provide basic GUI integration and user feedback for device discovery events, through the Desktop Notification specification. This will come as a PulseAudio module that will communicate through D-Bus to the notification system provided by the desktop environment. The module will be subscribing to the server to receive device discovery events and will be relaying that information to the user. If the user decides to take an action with respect to that device, this will be communicated back to the module (through the D-Bus Desktop Notification API) and the module will change the server state accordingly. I have split the development into the following tasks:

1. write a simple module to detect new devices;
2. add basic interaction with D-Bus through the Desktop Notification specification;
3. implement user feedback through DN;
4. implement persistence of user settings through the PulseAudio database;
5. improve the user intreface, from a usability point of view:
• should the user set a whole device as default or sinks/sources;
• should there be separate notifications for each of the sinks and sources of a device;
• what happens if a user ignores the notification etc.
6. adapt the zeroconf module to integrate with this module well (e.g. add authentication prompt);
7. inspect other available user feedback/notification systems, such as Ubuntu's, and extend the module to provide for those mechanisms as well;
8. make some adjustments to the existing GUIs (such as disabling volume controls when pass-through mode is enabled).

I will start coding in about a week and I will try to post updates on my progress weekly. The posts will not be aimed only for those who have a good understanding of PulseAudio; I will try to explain what I discover and learn so that others who would want to start coding for PulseAudio can read through the posts easily and understand how everything works.

I'm looking forward to this summer and to working on this project!

May 24, 2012 02:53 PM

The Magic Continues

Hot on the heals of the awesome PulseAudio 2.0 release, I am very please to announce Mageia 2! It's been a lot of very hard work and I inadvertently ended up doing a lot more than I had originally intended but I can't complain - while it was a lot of hard work and a massive time sink for the last few months, I certainly learned a great deal and feel I've contributed to a great user experience.So what makes Mageia magic? Well I like to think it's because we've got a great bunch of people involved - not only that, we have clear guidelines about how the whole project can never be own