Chasing a (possible) udev issue...

I have been chasing, what might possibly be, a problem in udev, and am using this blog post as a gathering point for evidence.

Because there is so little to be found on this issue, it is almost certainly a configuration issue in my own environment. Otherwise the Internet would be full of people seeing the same problems.

I also cannot guarantee that all of this evidence is related, nor can I be 100% certain that it is actually due to a problem with udev. All I can say for certain is that the evidence seems to be pointing towards an issue with device nodes not being created when they should be. Since udev is responsible for managing device nodes, it seems reasonable to start considering that udev may be at fault somehow.


First some background information. I am maintaining my own yocto based Linux distribution, currently working from the Pyro branch.

• bash version - 4.3.47(1) (x86_64)
• LVM version information:
• LVM Version: 2.02.166(2) (2016-09-26)
• Library Version: 1.02.135 (2016-09-26)
• Driver Version: 4.31.0
• udev version 232

The first problem...

When I attempt to create a new logical volume:

root@server:~# lvcreate -L1M -ntest.data vg00
Rounding up size to full physical extent 4.00 MiB
/dev/vg00/test.data: not found: device not cleared
Aborting. Failed to wipe start of new LV.

When you create a new logical volume with lvcreate, the first 4KiB needs to be zeroed out to avoid a potential hang when mounting the volume. It appears as if the device file (/dev/vg00/test.data) is not being created in time for the remaining lvcreate tasks to finish. The workaround is to use the -Zn argument to lvcreate and then do a manual zeroing with the dd command, like the following:

root@server:~# lvcreate -Zn -L1M -ntest.data vg00

Rounding up size to full physical extent 4.00 MiB
WARNING: Logical volume vg00/test.data is not zeroed.
Logical volume "test.data" created.

root@server:~# dd if=/dev/zero of=/dev/vg00/test.data bs=512 count=8

8+0 records in
8+0 records out
4096 bytes (4.1kB, 4.0 KiB) copied, 0.00415683 s, 985 kB/s

The second problem...

This one happens when I attempt to use bash process substitution. What I expect to see is something like the following:

root@server:~# echo <(true)
/dev/fd/63

What I actually see is the following:

root@server:~# echo <(true)
-sh: syntax error near unexpected token `('

The characters are absolutely identical in both cases - I have copied and pasted them in every way I know how. I have also tried the same command in a multitude of bash interpreters of various versions and it always works as expected.

Because bash process substitution depends on device nodes being created on the fly, this seems like it would be related to the LVM problem above.

Conclusions...

As I mentioned at the top, I am not 100% certain that this is all related, but it seems pretty suspicious. Two instances that rely on device files to be created on the fly, both seem to fail. Conversely, the boot process, which creates a lot of device nodes, seems to work just fine. I am also not able to find any smoking guns (or even faint whiffs) in the logs.

More to come...