224 lines · plain
1.. _development_early_stage:2 3Early-stage planning4====================5 6When contemplating a Linux kernel development project, it can be tempting7to jump right in and start coding. As with any significant project,8though, much of the groundwork for success is best laid before the first9line of code is written. Some time spent in early planning and10communication can save far more time later on.11 12 13Specifying the problem14----------------------15 16Like any engineering project, a successful kernel enhancement starts with a17clear description of the problem to be solved. In some cases, this step is18easy: when a driver is needed for a specific piece of hardware, for19example. In others, though, it is tempting to confuse the real problem20with the proposed solution, and that can lead to difficulties.21 22Consider an example: some years ago, developers working with Linux audio23sought a way to run applications without dropouts or other artifacts caused24by excessive latency in the system. The solution they arrived at was a25kernel module intended to hook into the Linux Security Module (LSM)26framework; this module could be configured to give specific applications27access to the realtime scheduler. This module was implemented and sent to28the linux-kernel mailing list, where it immediately ran into problems.29 30To the audio developers, this security module was sufficient to solve their31immediate problem. To the wider kernel community, though, it was seen as a32misuse of the LSM framework (which is not intended to confer privileges33onto processes which they would not otherwise have) and a risk to system34stability. Their preferred solutions involved realtime scheduling access35via the rlimit mechanism for the short term, and ongoing latency reduction36work in the long term.37 38The audio community, however, could not see past the particular solution39they had implemented; they were unwilling to accept alternatives. The40resulting disagreement left those developers feeling disillusioned with the41entire kernel development process; one of them went back to an audio list42and posted this:43 44 There are a number of very good Linux kernel developers, but they45 tend to get outshouted by a large crowd of arrogant fools. Trying46 to communicate user requirements to these people is a waste of47 time. They are much too "intelligent" to listen to lesser mortals.48 49(https://lwn.net/Articles/131776/).50 51The reality of the situation was different; the kernel developers were far52more concerned about system stability, long-term maintenance, and finding53the right solution to the problem than they were with a specific module.54The moral of the story is to focus on the problem - not a specific solution55- and to discuss it with the development community before investing in the56creation of a body of code.57 58So, when contemplating a kernel development project, one should obtain59answers to a short set of questions:60 61 - What, exactly, is the problem which needs to be solved?62 63 - Who are the users affected by this problem? Which use cases should the64 solution address?65 66 - How does the kernel fall short in addressing that problem now?67 68Only then does it make sense to start considering possible solutions.69 70 71Early discussion72----------------73 74When planning a kernel development project, it makes great sense to hold75discussions with the community before launching into implementation. Early76communication can save time and trouble in a number of ways:77 78 - It may well be that the problem is addressed by the kernel in ways which79 you have not understood. The Linux kernel is large and has a number of80 features and capabilities which are not immediately obvious. Not all81 kernel capabilities are documented as well as one might like, and it is82 easy to miss things. Your author has seen the posting of a complete83 driver which duplicated an existing driver that the new author had been84 unaware of. Code which reinvents existing wheels is not only wasteful;85 it will also not be accepted into the mainline kernel.86 87 - There may be elements of the proposed solution which will not be88 acceptable for mainline merging. It is better to find out about89 problems like this before writing the code.90 91 - It's entirely possible that other developers have thought about the92 problem; they may have ideas for a better solution, and may be willing93 to help in the creation of that solution.94 95Years of experience with the kernel development community have taught a96clear lesson: kernel code which is designed and developed behind closed97doors invariably has problems which are only revealed when the code is98released into the community. Sometimes these problems are severe,99requiring months or years of effort before the code can be brought up to100the kernel community's standards. Some examples include:101 102 - The Devicescape network stack was designed and implemented for103 single-processor systems. It could not be merged into the mainline104 until it was made suitable for multiprocessor systems. Retrofitting105 locking and such into code is a difficult task; as a result, the merging106 of this code (now called mac80211) was delayed for over a year.107 108 - The Reiser4 filesystem included a number of capabilities which, in the109 core kernel developers' opinion, should have been implemented in the110 virtual filesystem layer instead. It also included features which could111 not easily be implemented without exposing the system to user-caused112 deadlocks. The late revelation of these problems - and refusal to113 address some of them - has caused Reiser4 to stay out of the mainline114 kernel.115 116 - The AppArmor security module made use of internal virtual filesystem117 data structures in ways which were considered to be unsafe and118 unreliable. This concern (among others) kept AppArmor out of the119 mainline for years.120 121In each of these cases, a great deal of pain and extra work could have been122avoided with some early discussion with the kernel developers.123 124 125Who do you talk to?126-------------------127 128When developers decide to take their plans public, the next question will129be: where do we start? The answer is to find the right mailing list(s) and130the right maintainer. For mailing lists, the best approach is to look in131the MAINTAINERS file for a relevant place to post. If there is a suitable132subsystem list, posting there is often preferable to posting on133linux-kernel; you are more likely to reach developers with expertise in the134relevant subsystem and the environment may be more supportive.135 136Finding maintainers can be a bit harder. Again, the MAINTAINERS file is137the place to start. That file tends to not always be up to date, though,138and not all subsystems are represented there. The person listed in the139MAINTAINERS file may, in fact, not be the person who is actually acting in140that role currently. So, when there is doubt about who to contact, a141useful trick is to use git (and "git log" in particular) to see who is142currently active within the subsystem of interest. Look at who is writing143patches, and who, if anybody, is attaching Signed-off-by lines to those144patches. Those are the people who will be best placed to help with a new145development project.146 147The task of finding the right maintainer is sometimes challenging enough148that the kernel developers have added a script to ease the process:149 150::151 152 .../scripts/get_maintainer.pl153 154This script will return the current maintainer(s) for a given file or155directory when given the "-f" option. If passed a patch on the156command line, it will list the maintainers who should probably receive157copies of the patch. This is the preferred way (unlike "-f" option) to get the158list of people to Cc for your patches. There are a number of options159regulating how hard get_maintainer.pl will search for maintainers; please be160careful about using the more aggressive options as you may end up including161developers who have no real interest in the code you are modifying.162 163If all else fails, talking to Andrew Morton can be an effective way to164track down a maintainer for a specific piece of code.165 166 167When to post?168-------------169 170If possible, posting your plans during the early stages can only be171helpful. Describe the problem being solved and any plans that have been172made on how the implementation will be done. Any information you can173provide can help the development community provide useful input on the174project.175 176One discouraging thing which can happen at this stage is not a hostile177reaction, but, instead, little or no reaction at all. The sad truth of the178matter is (1) kernel developers tend to be busy, (2) there is no shortage179of people with grand plans and little code (or even prospect of code) to180back them up, and (3) nobody is obligated to review or comment on ideas181posted by others. Beyond that, high-level designs often hide problems182which are only revealed when somebody actually tries to implement those183designs; for that reason, kernel developers would rather see the code.184 185If a request-for-comments posting yields little in the way of comments, do186not assume that it means there is no interest in the project.187Unfortunately, you also cannot assume that there are no problems with your188idea. The best thing to do in this situation is to proceed, keeping the189community informed as you go.190 191 192Getting official buy-in193-----------------------194 195If your work is being done in a corporate environment - as most Linux196kernel work is - you must, obviously, have permission from suitably197empowered managers before you can post your company's plans or code to a198public mailing list. The posting of code which has not been cleared for199release under a GPL-compatible license can be especially problematic; the200sooner that a company's management and legal staff can agree on the posting201of a kernel development project, the better off everybody involved will be.202 203Some readers may be thinking at this point that their kernel work is204intended to support a product which does not yet have an officially205acknowledged existence. Revealing their employer's plans on a public206mailing list may not be a viable option. In cases like this, it is worth207considering whether the secrecy is really necessary; there is often no real208need to keep development plans behind closed doors.209 210That said, there are also cases where a company legitimately cannot211disclose its plans early in the development process. Companies with212experienced kernel developers may choose to proceed in an open-loop manner213on the assumption that they will be able to avoid serious integration214problems later. For companies without that sort of in-house expertise, the215best option is often to hire an outside developer to review the plans under216a non-disclosure agreement. The Linux Foundation operates an NDA program217designed to help with this sort of situation; more information can be found218at:219 220 https://www.linuxfoundation.org/nda/221 222This kind of review is often enough to avoid serious problems later on223without requiring public disclosure of the project.224