Linux 2.6.28 kernel
As most of us celebrate the holidays with friends and family, some of us tinker with new Kernels. Thanks to Linus fof the latest major release of the Linux Kernel (2.6.28)
Here are some of the more notable (at least my favorites) improvements that are packed along with this release.
Ext4
The backwards-compatible replacement of Ext3 has been declared as stable. Ext4 gives you larger file-system and file sizes, faster I/O, better journaling, and it can defragment your drive on the fly.
In particular, its delayed allocation functionality greatly improves hard disk write performance. Try it yourself on your servers, you’ll be impressed. In addition, since Ext4 can handle up to 1024 petabytes per volume, I expect Ext4 and Sun’s ZFS are going to be fighting it out for top server file-system for the next ten-years.
Disk Shock Protection
ATA/ATAPI-7 specifies the IDLE IMMEDIATE command with unload feature. Issuing this command should cause the drive to switch to idle mode and unload disk heads. This feature is being used in modern laptops in conjunction with accelerometers and appropriate software to implement a shock protection facility. The idea is to stop all I/O operations on the internal hard drive and park its heads on the ramp when critical situations are anticipated.
For each ATA device, Linux 2.6.28 adds the file /sys/block/*/device/unload_heads. Writing an integer value to this file will take the heads of the respective drive off the platter and defer all I/O operations for the specified number of milliseconds. When the timeout expires normal operation will be resumed. The maximal value accepted for a timeout is 30000 milliseconds. However, there are some hard drives that only comply with an earlier version of the ATA standard, but do support the feature nonetheless. Unfortunately, there is no safe way Linux can detect these devices, so you won’t be able to write to the unload_heads attribute. If you know that your device really does support the unload feature (for instance, because the vendor of your laptop or the hard drive itself told you so), then you can tell the kernel to enable the usage of this feature for that drive by writing the special value -1 to the unload_heads attribute. See this page for information about Linux support of the hard disk active protection system as implemented in IBM/Lenovo Thinkpads.
GEM Memory Manager for Graphics
In the last decade graphics hardware has evolved at an astounding pace, and it’s expected to improve even more in the future. Modern GPUs have a lot of processing power -more than the most powerful CPU in some specialized workloads- that traditionally has only been used by specialized applications using opengl/directx, like games and 3D design apps; the 2D desktop implementations that are commonly found in computers kept using this modern graphic hardware in the same way they used the old graphics hardware which started the “desktop revolution” in the 80-90’s, ie. inefficiently. There’s a lot of GPU power that don’t get used unless you run a game. On the other hand, the Linux/FOSS graphic stack is far from perfect, even for the traditional graphic stack design. To start with, there’re several drivers fighting to access the same resource (the graphics card): The fb-based console, the in-kernel DRM driver, the X.org userspace 2D driver…this situation leads to all class of problems and artifacts and suboptimal performance.
There has been a lot of work in the latest years to modernize the Linux graphics stack so that it’s both well designed and also ready to use the full power of modern and future GPUs. In 2.6.28, Linux is adding one of the most important pieces of the stack: A memory manager for the GPU memory, called GEM (”Graphic Execution Manager”). The purpose is to have a central manager for buffer object placement, caching, mapping and synchronization. It speeds up some benchmarks by 50%. On top of GEM are being built a lot of improvementes to the graphic stack: Kernel Modesetting, DRI2, UXA (a EXA implementation based in GEM). The Linux/FOSS graphics stack will be finally unified and optimally coupled.
All this new code has been delayed for a long time, because there was a competing memory manager, called TTM, which was almost merged in the kernel in 2.6.24 or so, until the Intel people came up with the first versions of the GEM memory manager. People decided it was better than TTM, and it was considered neccesary to delay the merge to stabilize GEM and rewrite the other features to work with GEM, not TTM. Hence, this first version of GEM works only with the i915 driver, and support on the X.org side is implemented only in the version 2.5.0 of the driver. Preliminary GEM support for other drivers is already in development and will be merged in future releases.
Boot Tracer
The purpose of this tracer is to helps developers to optimize boot times: it records the timings of the initcalls. Its aim is to be parsed by the scripts/bootgraph.pl tool to produce graphics about boot inefficiencies, giving a visual representation of the delays during initcalls. Users need to enable CONFIG_BOOT_TRACER, boot with the “initcall_debug” and “printk.time=1″ parameters, and run “dmesg | perl scripts/bootgraph.pl > output.svg” to generate the final data.
Network improvements
Ultra Wide Band” (UWB) is a high-bandwidth, low-power, point-to-point radio technology using a wide spectrum (3.1-10.6GHz). It is optimized for in-room use (480Mbps at 2 meters, 110Mbps at 10m). It serves as the transport layer for other protocols, such as Wireless USB, WiMedia Link Protocol (Ethernet/IP over UWB) and, in the future, Bluetooth and 1394. Linux 2.6.28 adds code to implement a Ultra Wide Band stack, as well as drivers for the the USB based UWB radio controllers defined in the Wireless USB 1.0 specification (including Wireless USB host controller and an Intel WiNET controller).
Memory management Scalability improvements
Systems with a lot of memory have lots (millions) of pages. When the replacement algorithm has to search candidate pages to be swapped, it has to search between all the pages, and in big systems this can take too much time. In 2.6.28, the file-backed pages (pages that belong to some file on the disk) and the anonymous pages (pages that are not part of any file, like ie. pages obtained with malloc, which need to be written to swap before being evicted) are put in two different lists, unlike previous releases, that used a single list. The algorithms can decide to look into only one of those lists without needing to look in the other. Additionally, there’re pages that cannot be deleted from memory, for example because they’re mlock()’ed, or because they belong to a ramfs filesystem. Those pages are put into a special third list, which won’t be searched at all by the algorithms because they can not be evicted.
Here are some of the more notable (at least my favorites) improvements that are packed along with this release.
Ext4
The backwards-compatible replacement of Ext3 has been declared as stable. Ext4 gives you larger file-system and file sizes, faster I/O, better journaling, and it can defragment your drive on the fly.
In particular, its delayed allocation functionality greatly improves hard disk write performance. Try it yourself on your servers, you’ll be impressed. In addition, since Ext4 can handle up to 1024 petabytes per volume, I expect Ext4 and Sun’s ZFS are going to be fighting it out for top server file-system for the next ten-years.
Disk Shock Protection
ATA/ATAPI-7 specifies the IDLE IMMEDIATE command with unload feature. Issuing this command should cause the drive to switch to idle mode and unload disk heads. This feature is being used in modern laptops in conjunction with accelerometers and appropriate software to implement a shock protection facility. The idea is to stop all I/O operations on the internal hard drive and park its heads on the ramp when critical situations are anticipated.
For each ATA device, Linux 2.6.28 adds the file /sys/block/*/device/unload_heads. Writing an integer value to this file will take the heads of the respective drive off the platter and defer all I/O operations for the specified number of milliseconds. When the timeout expires normal operation will be resumed. The maximal value accepted for a timeout is 30000 milliseconds. However, there are some hard drives that only comply with an earlier version of the ATA standard, but do support the feature nonetheless. Unfortunately, there is no safe way Linux can detect these devices, so you won’t be able to write to the unload_heads attribute. If you know that your device really does support the unload feature (for instance, because the vendor of your laptop or the hard drive itself told you so), then you can tell the kernel to enable the usage of this feature for that drive by writing the special value -1 to the unload_heads attribute. See this page for information about Linux support of the hard disk active protection system as implemented in IBM/Lenovo Thinkpads.
GEM Memory Manager for Graphics
In the last decade graphics hardware has evolved at an astounding pace, and it’s expected to improve even more in the future. Modern GPUs have a lot of processing power -more than the most powerful CPU in some specialized workloads- that traditionally has only been used by specialized applications using opengl/directx, like games and 3D design apps; the 2D desktop implementations that are commonly found in computers kept using this modern graphic hardware in the same way they used the old graphics hardware which started the “desktop revolution” in the 80-90’s, ie. inefficiently. There’s a lot of GPU power that don’t get used unless you run a game. On the other hand, the Linux/FOSS graphic stack is far from perfect, even for the traditional graphic stack design. To start with, there’re several drivers fighting to access the same resource (the graphics card): The fb-based console, the in-kernel DRM driver, the X.org userspace 2D driver…this situation leads to all class of problems and artifacts and suboptimal performance.
There has been a lot of work in the latest years to modernize the Linux graphics stack so that it’s both well designed and also ready to use the full power of modern and future GPUs. In 2.6.28, Linux is adding one of the most important pieces of the stack: A memory manager for the GPU memory, called GEM (”Graphic Execution Manager”). The purpose is to have a central manager for buffer object placement, caching, mapping and synchronization. It speeds up some benchmarks by 50%. On top of GEM are being built a lot of improvementes to the graphic stack: Kernel Modesetting, DRI2, UXA (a EXA implementation based in GEM). The Linux/FOSS graphics stack will be finally unified and optimally coupled.
All this new code has been delayed for a long time, because there was a competing memory manager, called TTM, which was almost merged in the kernel in 2.6.24 or so, until the Intel people came up with the first versions of the GEM memory manager. People decided it was better than TTM, and it was considered neccesary to delay the merge to stabilize GEM and rewrite the other features to work with GEM, not TTM. Hence, this first version of GEM works only with the i915 driver, and support on the X.org side is implemented only in the version 2.5.0 of the driver. Preliminary GEM support for other drivers is already in development and will be merged in future releases.
Boot Tracer
The purpose of this tracer is to helps developers to optimize boot times: it records the timings of the initcalls. Its aim is to be parsed by the scripts/bootgraph.pl tool to produce graphics about boot inefficiencies, giving a visual representation of the delays during initcalls. Users need to enable CONFIG_BOOT_TRACER, boot with the “initcall_debug” and “printk.time=1″ parameters, and run “dmesg | perl scripts/bootgraph.pl > output.svg” to generate the final data.
Network improvements
Ultra Wide Band” (UWB) is a high-bandwidth, low-power, point-to-point radio technology using a wide spectrum (3.1-10.6GHz). It is optimized for in-room use (480Mbps at 2 meters, 110Mbps at 10m). It serves as the transport layer for other protocols, such as Wireless USB, WiMedia Link Protocol (Ethernet/IP over UWB) and, in the future, Bluetooth and 1394. Linux 2.6.28 adds code to implement a Ultra Wide Band stack, as well as drivers for the the USB based UWB radio controllers defined in the Wireless USB 1.0 specification (including Wireless USB host controller and an Intel WiNET controller).
Memory management Scalability improvements
Systems with a lot of memory have lots (millions) of pages. When the replacement algorithm has to search candidate pages to be swapped, it has to search between all the pages, and in big systems this can take too much time. In 2.6.28, the file-backed pages (pages that belong to some file on the disk) and the anonymous pages (pages that are not part of any file, like ie. pages obtained with malloc, which need to be written to swap before being evicted) are put in two different lists, unlike previous releases, that used a single list. The algorithms can decide to look into only one of those lists without needing to look in the other. Additionally, there’re pages that cannot be deleted from memory, for example because they’re mlock()’ed, or because they belong to a ramfs filesystem. Those pages are put into a special third list, which won’t be searched at all by the algorithms because they can not be evicted.