login
Search
Should the audio driver be smart enough to know that the codec needs to remain powered up during every opportunistic suspend? Or only during opportunistic suspends while a call is in progress? Does it know when a call is in progress? Does Android use forced suspends? If it does, are they supposed to shut down the codec? Alan Stern --
Perhaps you could infer that from the audio routing setup? I don't know No. -- Matthew Garrett | mjg59@srcf.ucam.org --
Yes, you can providing we extract a bit more information from the machine drivers and/or userspace - we already do our power management based on paths. The implementation I'd do here is to provide a facility for marking some of the path endpoints as suspend insensitive then on suspend leave those endpoints in their current state, force the other endpoints off temporarily and re-run the standard power checks. One thing I'll need to have a bit of a think about is if we need to provide the ability to offer control to userspace for the suspend state of path endpoints separately to their runtime state; I suspect the answer is yes. --
Hi Mark, I confess I've completely lost track of (a) what problem you are trying to solve, and (b) how this might relate to some change that you'd like to see in the suspend block API. Could you do a quick summary and recap? I've gone over the entire thread, and it's still not clear what change you're advocating for in suspend blockers. Thanks, regards, - Ted --
The issue isn't suspend blockers, it's the opportunistic suspend stuff that goes along with them. When that is in use the system suspends vastly more aggressively, including in situations where a runtime PM based approach like mainline had been adopting would not suspend since some devices still need to be active, the classic case being keeping the audio subsystem and baseband live when in a phone call. This problem did not appear to have been considered as things stood. I'm not really advocating a change in what's there. What I'm looking for is some sort of agreement as to how subsystems and drivers that need to not act on suspend requests from the core in these situations should do that. If there is a generic solution it'd probably be an additional mostly orthogonal interface rather than a change to what's being proposed here. What we look like we're converging on is a subsystem/driver local solution since it doesn't look like a terribly widespread problem. That's totally OK, it's just that as I have said I don't want to go off and do that without the general PM community being aware of it so we avoid anyone running into nasty surprises further down the line. --
To me, the above may be summarized that in your opinion some components of the system will generally need to stay powered when it's suspended opportunistically, so we need an interface to specify which components they are. Is that correct? Rafael --
Yes, though I think I'd be inclined to treat the problem orthogonally to opportunistic suspend to allow more flexibility in use and since treating it as part of opportunistic suspend would imply that there's some meaningful difference between the end result of that and a manual suspend which AIUI there isn't. --
On Wed, May 5, 2010 at 2:57 PM, Mark Brown I'd agree with this. This is one of the reasons I haven't felt opportunistic suspend is a big departure from other work here -- there are always some cases in which drivers will need to keep external resources active even when suspended. Also, as I mentioned earlier, we (Android at least, but hopefully this is non-controversial) would always like drivers to put everything in the lowest possible power state they can get away with. Every little savings adds up. Brian --
No, there's no such difference. So, gnerenally, we may need a mechanism to specify which components of the system need to stay powered while the whole system is suspended (in addition to wakeup devices, that is). That certainly I can agree with. I'm not sure, however, in what way this is relevant to the $subject patchset. Rafael --
The patch set essentially makes using a full system suspend as the lowest power state for runtime PM part of the standard Linux power management toolkit which means that it's no longer clear as it used to be that suspend is an instruction to cease all activity and go into a minimal power state if the device is not a wake source. In the primary existing application this change interoperates very poorly with at least the current audio subsystem since that handles suspend by ceasing all activity and powering as much as it can off, which is sensible for manual only suspends but highly undesirable for opportunistic suspend in phones. We should therefore have some idea how this and any other affected areas are supposed to work. As I said in my reply to Ted earlier I think we may actually be converging on not worrying too much about it at the global level and doing subsystem specific things to discover and handle affected cases, at least for the time being. Ideally we'd have something standard to hook into but no subsystems apart from audio have actually been identified as being affected so it's not clear to me that a general solution is going to be worth the effort, if there's no actual users it may just confuse people by adding yet more power managment stuff to learn. --
I don't see why, really. This patchset doesn't change the meaning of suspend at all, it only changes the way in which suspend is triggered (if specifically set up that way). Even without the patchset you may implement a power manager in user space that will suspend the system whenever it thinks it's You said that there's no fundamental difference between manual and opportunistic suspend. It only matters what _you_ are going to use suspend for. I agree that at the moment it's not suitable for aggressive power management in phones because of the audio problem, but that applies to "manual" as well as to "opportunistic" suspend. You're saying that suspend is not suitable for one particular purpose in its current form, which is entirely correct, but that doesn't imply that the patchset is wrong. Rafael --
...on the other hand there's exactly one existing application for this, and that's the one that's most likely to run into problems since it's a phone OS and aggressive power management is pretty important for phones. Merging a feature into mainline makes it much more something which one would expect to play nicely with the rest of the kernel - if it's something that isn't part of the standard kernel or userspaces it's much less surprising that additional changes may be required to produce a As I keep saying I agree that merging this is reasonable given the additional power savings it brings in practical systems today. As I also keep saying I do want to have some understanding about what the story is for dealing with the problems so that people can easily use this feature out of the box. Like I say, my current impression is that the best approach is for affected subsystems or drivers to implement a custom solution - does that match your understanding and that of the other PM maintainers? --
I agree that this appears to be the best approach for the time being, although I can only speak for myself. Rafael --
One thing that's been mentioned before is that some vendors can swap the kernel much more easily than they can swap userspace, with the added constraint that they have little control over the bootloader. So if there's a userspace interface, it may well also be helpful to provide a kernel hook to allow the platform setup code to configure things. But otherwise, I think that'd work for audio - the only question is whether we need it to be more generic. -- Matthew Garrett | mjg59@srcf.ucam.org --
Oh, there's definitely going to be a platform-specific hook to configure things. The platform needs to specify how the devices that make up the audio subsystem are interconnected on the board, and that's done in kernel since it depends on the hardware and can potentially lead to Agreed. --
I think for simplicity and maximum flexibility we should just ignore the reason for suspend here, if I'm adding support for this in the drivers the suspend reason is not going to be terribly useful and it makes the feature more widely useful should people come up with other applications. Note that we already have advanced runtime PM for embedded audio which will drive modern CODECs into full off state when the device is idle anyway, independently of any suspend or runtime PM that the system does. Older CODECs need their bias levels maintaining but we can ignore the Not at present. This is relatively straightforward to add within ASoC, especially for analogue links, if not completely trivial. Digital basebands are a bit more fun but not insurmountable (and need cleaning up anyway, there's some ongoing work which should hit soon). We currently drive everything into off state when we're told to suspend since it's the right thing to do for most non-phone devices. We'll still need to drive some of the system into an off state (eg, some of Pass, but it's not been an issue users have raised with me - but then we can't tell why the suspend happens and need to handle opportunistic I suspect it's a non-issue; if we're not actually using audio then ASoC will drive a modern CODEC into power off anyway as part of its runtime PM. However, I believe some non-Android phone stacks are doing something along that line. --
From my point of view, the codec should be powered up while audio is playing and powered down when it's not -- that's the approach I've taken on MSM, regardless of suspend state. I don't view suspend as something that stops audio playback in progress. Sort of similar to how the modem doesn't power off down when the system is in suspend, etc. In effect, we have followed a sort of runtime pm model of clocking/powering peripherals at the lowest possible level at all times, which is why I've never viewed suspend_block as somehow competing with runtime pm -- we always want all drivers/peripherals in the lowers power state. Sometimes the lowest power state possible is higher than "full suspend" (for example when we can't meet latency requirements on audio on 7201A) and suspend_blockers cover that case. Brian --
This is what we've got already with Linux - we've had rather aggressive runtime PM for embedded audio as standard for years now. It's only very recently that mobile hardware has got to the point where you can actually totally kill the power without introducing problems but it's This really does depend heavily on system design and the intended function of the suspend on the part of the initiator. In many systems suspend will remove sufficient power to stop audio running even if it were desired, and there's the idea with existing manually initiated suspends that the system should stop what it's doing and go into a low power, low responsiveness state. Some systems will initiate a suspend with active audio paths (especially those to or from the AP) which they Right, and this will work well on systems like your current devices because the problem hardware is not directly visible to the AP (the actual audio hardware is hidden behind the DSP which functions as autonomously as the modem does) so it's not affected by the Linux suspend process in the first place. It causes confusion when the power for the audio device is managed by Linux since the opportunistic suspend will include suspending the audio device and so the code handling the In this situation there's no physical reason why the lower power state is not achievable so blocking the suspend would just waste power, it's just that our way of getting into it is creating some confusion. --
On Wed, May 5, 2010 at 4:40 PM, Mark Brown You can support both in the same driver in some cases (without a switch). If the hardware cannot wake the system when it needs more audio buffers, then the driver needs to block suspend while audio is playing. Since suspend blockers only block opportunistic suspend, the driver can also implement suspend to stop audio playback and it will only be called for forced suspend. -- Arve Hjønnevåg --
As discussed elsewhere in the thread a suspend blocker is not desirable here - the AP is not doing anything useful on a voice call so blocking the suspend will just waste power unless runtime PM is good enough to mean opportunistic suspend isn't adding anything anyway. It will avoid the immediate issue with loosing audio but it's not really what we want to happen. --
I was talking about audio from the AP. Why would you ever turn off another core's audio path on suspend? -- Arve Hjønnevåg --
This goes back to the thing about a full system suspend being a sledgehammer which doesn't give enough information about what's going on when it's used like this. As discussed we need a way to know that the connections involved are able to stay live during suspend (on a large proportion of systems suspend is going to mean that the relevant bits of the board will loose power so need to be shut down) and that that the intention of the user is that they should do so (this isn't clear in the general system, especially not if the suspend is initiated manually). With a runtime PM approach this is trivial - we just turn off anything that isn't in use at the current time. I'll need to extend ASoC to add information about what to do on suspend to the existing power data. --
This sounds like it may be something unique to your board/product. Or am I missing something? One of the challenges with PM in the embedded world is that everybody seems to have slightly different assumptions, and hardware that doesn't behave the same way. More than once this discussion has wandered off into the weeds wrt to whether this patch series is ready to be merged, since there are so many drivers blocked on it.... -- Ted --
I suspect you're missing something here - I'm approaching this as one of the maintainers of the embedded audio subsystem for the kernel. I need to worry about any system running Linux which has an embedded style audio subsystem. The problem might be unique to audio, but I can't say This isn't really a problem for audio - we've already got a subsystem which copes well with pretty much all systems and does runtime PM that's equivalent to suspending already, which is half the problem here. If we Once more, my main objective here has been to make sure that when this is merged we've got a joined up story about how people think this hangs together, which I think we have now. As I say now that we have that understanding I don't see any reason to block merge. It's unfortunate that I only noticed that this was actually wakelocks very late in the day but I think I can get an implementation which handles paths that ignore suspends done quickly. --
On Fri, May 7, 2010 at 5:25 AM, Mark Brown Do you mean an implementation of embedded linux audio or an implementation of suspend blockers "which handles paths that ignore suspends"? I assume you mean the former, but maybe I misunderstand. We've done a lot of work around multicore SoCs where the baseband generally owns a lot of the audio routing and so on, but we do as general policy not disable audio, codecs, and amps while playback or record is underway. I suppose for environments more like a pc laptop where the expectation is the world stops when you close the lid a different policy would be preferable. We typically fire up codecs and amps when playback starts (if they were off), and usually set a timer for a couple seconds after playback stops to spin them down (since you tend to have to delay while they're powering up to avoid pops, etc, and if the system just played a short sound there's a reasonable chance it'll follow that with another). Apologies about the name confusion -- last year when we first started pushing these patches out there was much dislike for the name wakelock and after a lot of back and forth suspend_block ended up as the least hated of the suggested alternatives (I kinda like "wakelock" as a name, but maybe that's just brain rot after dealing with 'em for four years now). Brian --
I mean that I will extend the embedded audio subsystem that the kernel already has (ASoC, in sound/soc) to support ignoring suspends for some audio paths so that they can be kept up during suspend. This will be primarily intended for use with opportunistic suspend but not specific to it. I have no intention of touching suspend blockers, except in that some of the drivers I'm responsible for should probably use suspend blockers for Multicore isn't really the term here - obviously on something like the OMAP4 or the current nVidia devices you've got a multicore AP but may or may not have an integrated baseband. The distinction is down to how much visibility the AP has of the audio hardware, which is in general Yeah, I know. I have been keeping an eye on the Android stuff, though Yes, quite. It all depends on what the intention of the user is when they do a suspend. Clearly if the suspend was initiated automatically because the system is apparently idle the intention is different to if This is exactly what ASoC does - the two biggest wins it offers are that it allows reuse of the code specific to a given piece of silicon on all boards using that silicon and that it provides automatic runtime power management in the core by keeping track of which audio paths are live. There is the slight wrinkle that you don't really want to *fully* power down devices that don't have ground referenced outputs since they take a relatively long time to bring their reference level up without audible artifacts in the output. --
Just for the record I've now done an implementation of this which should show up in -next when it's rebuilt and 2.6.35. It's not the most thoroughly tested code I've ever written but I'm fairly happy it'll do the right thing, especially for analogue basebands. The functionality needs to be explicitly requested by machine drivers so there should be no impact on systems using suspend in a more standard fashion. This means that there should be even less of an issue merging this from an audio point of view. --
Are you trying to use the same driver on systems that can support audio while suspend and on systems where this is impossible? If audio cannot work while suspended, supporting opportunistic suspend is easy. You block suspend while audio is active. If the hardware does support some audio paths while suspended, I'll OK, but is this at all related to opportunistic suspend? -- Arve Hjønnevåg --
Yes, each chip in the audio subsystem gets its own device driver. All Linux systems using a given CODEC share the device specific driver code (modulo out of tree code, obviously). Further, the power management is mostly abstracted out of the individual device drivers into the embedded audio core so the individual drivers don't really take any decisions this all comes down to the intention of the user when they initiated the Well, opportunistic suspend is essentially blurring the lines between a suspend and runtime PM - it's using suspend as part of the automatic drive down into the lowest possible power state. I'm just saying that in a pure runtime PM model this issue doesn't come up since the suspend semantic is much clearer. --