Message ID | 20240425055231.668897-1-leimaohui@fujitsu.com |
---|---|
State | New |
Headers | show |
Series | [v3] libbsd: Fix conflict error when enable multilib. | expand |
On Thu, 25 Apr 2024 at 07:52, leimaohui via lists.openembedded.org
<leimaohui=fujitsu.com@lists.openembedded.org> wrote:
> - After oe_multilib_header on cdefs.h, the path of cdefs-64.h and cdefs-32.h in cdefs.h need to be corrected. Please reference to https://man.archlinux.org/man/libbsd.7 for details.
I looked at this in a local build, and I actually don't think they
need to be corrected. Without the sed command, there is #include
<bsd/sys/cdefs-32.h>, which is fine, as it's relative to /usr/include,
and unambigious.
After the sed command, there is #include <sys/cdefs-32.h> which
requires special non-default arrangements in the build system,
specifically, using libbsd in overlay mode with special compiler flags
as described in the man page you refer to. But that's not the only way
to use libbsd, and the other way won't work.
So what is the issue that happens if there is no sed, and only
oe_multilib_header?
Alex
Hi, Alex > After the sed command, there is #include <sys/cdefs-32.h> which requires > special non-default arrangements in the build system, specifically, using libbsd in > overlay mode with special compiler flags as described in the man page you refer > to. But that's not the only way to use libbsd, and the other way won't work. Reference to the man page, libbsd recommends overlay mode. "The library can be used in an overlay mode, which is the preferred way, so that the code is portable and requires no modification to the original BSD code......." From the build log and the source code of libbsd, I found that libbsd does provide header file in overlay mode by default as the man page says. ----------------------------------------------------------------------- $ ls tmp/work/cortexa57-poky-linux/libbsd/0.12.2/image/usr/lib64/pkgconfig/ libbsd-ctor.pc libbsd-overlay.pc libbsd.pc $ grep include tmp/work/cortexa57-poky-linux/libbsd/0.12.2/temp/log.do_compile -r ...... -I.. -isystem ../../libbsd-0.12.2/include/bsd/...... ...... -I.. -isystem ../../libbsd-0.12.2/include/bsd/...... ...... $ vim tmp/work/cortexa57-poky-linux/libbsd/0.12.2/libbsd-0.12.2/include/bsd/stringlist.h #ifdef LIBBSD_OVERLAY #include <sys/cdefs.h> #else #include <bsd/sys/cdefs.h> #endif #include <sys/types.h> ---------------------------------------------------------------------- And I Investigated several OSS(e.g. shadow, mg, libxdmcp) that depend on libbsd. From the source code of theirs. I found that they does used libbsd in overlay mode by default. Such as shadow: ----------------------------------------------------------------------- vim tmp/work/cortexa57-poky-linux/shadow/4.15.0/shadow-4.15.0/lib/readpassphrase.h ...... #ifdef LIBBSD_OVERLAY #include <sys/cdefs.h> #endif ...... ----------------------------------------------------------------------- The same as mg : ----------------------------------------------------------------------- $ vim tmp/work/cortexa57-poky-linux/mg/20230501/git/CMakeLists.txt if(CMAKE_SYSTEM_NAME MATCHES "Linux") pkg_check_modules (BSD REQUIRED libbsd-overlay) link_directories (${BSD_LIBRARY_DIRS}) string (REPLACE ";" " " LIBBSD_FLAGS "${BSD_CFLAGS}") target_link_libraries (mg ${BSD_LIBRARIES}) set (CMAKE_C_FLAGS "-Wall -DREGEX -D_GNU_SOURCE ${LIBBSD_FLAGS} ${NCURSES_FLAGS} -L${NCURSES_LIBRARY_DIRS}") else() set (CMAKE_C_FLAGS "-Wall -DREGEX ${LIBBSD_FLAGS} ${NCURSES_FLAGS} -L${NCURSES_LIBRARY_DIRS}") endif() ----------------------------------------------------------------------- So, I think the header file after oe_multilib_header should be in overlay mode as the other header files in libbsd. Of course, If added "#ifdef LIBBSD_OVERLAY" into the header file after oe_multilib_header, libbsd will have a better compatibility. What do you think? Best regards Lei > -----Original Message----- > From: Alexander Kanavin <alex.kanavin@gmail.com> > Sent: Thursday, April 25, 2024 4:08 PM > To: Lei, Maohui <leimaohui@fujitsu.com> > Cc: openembedded-core@lists.openembedded.org > Subject: Re: [OE-core][PATCH v3] libbsd: Fix conflict error when enable multilib. > > On Thu, 25 Apr 2024 at 07:52, leimaohui via lists.openembedded.org > <leimaohui=fujitsu.com@lists.openembedded.org> wrote: > > - After oe_multilib_header on cdefs.h, the path of cdefs-64.h and cdefs-32.h in > cdefs.h need to be corrected. Please reference to > https://man.archlinux.org/man/libbsd.7 for details. > > I looked at this in a local build, and I actually don't think they need to be > corrected. Without the sed command, there is #include <bsd/sys/cdefs-32.h>, > which is fine, as it's relative to /usr/include, and unambigious. > > After the sed command, there is #include <sys/cdefs-32.h> which requires > special non-default arrangements in the build system, specifically, using libbsd in > overlay mode with special compiler flags as described in the man page you refer > to. But that's not the only way to use libbsd, and the other way won't work. > > So what is the issue that happens if there is no sed, and only > oe_multilib_header? > > Alex
On Fri, 26 Apr 2024 at 08:55, Maohui Lei (Fujitsu) <leimaohui@fujitsu.com> wrote: > From the build log and the source code of libbsd, I found that libbsd does provide header file in overlay mode by default as the man page says. > ----------------------------------------------------------------------- > $ ls tmp/work/cortexa57-poky-linux/libbsd/0.12.2/image/usr/lib64/pkgconfig/ > libbsd-ctor.pc libbsd-overlay.pc libbsd.pc > $ grep include tmp/work/cortexa57-poky-linux/libbsd/0.12.2/temp/log.do_compile -r > ...... -I.. -isystem ../../libbsd-0.12.2/include/bsd/...... > ...... -I.. -isystem ../../libbsd-0.12.2/include/bsd/...... > ...... > $ vim tmp/work/cortexa57-poky-linux/libbsd/0.12.2/libbsd-0.12.2/include/bsd/stringlist.h > #ifdef LIBBSD_OVERLAY > #include <sys/cdefs.h> > #else > #include <bsd/sys/cdefs.h> > #endif > #include <sys/types.h> > ---------------------------------------------------------------------- The man page does not say overlay mode is default, it only says that it's recommended. And indeed the above shows that you need to opt into it by defining LIBBSD_OVERLAY. Let's see how example projects handle it. > And I Investigated several OSS(e.g. shadow, mg, libxdmcp) that depend on libbsd. From the source code of theirs. I found that they does used libbsd in overlay mode by default. > Such as shadow: > ----------------------------------------------------------------------- > vim tmp/work/cortexa57-poky-linux/shadow/4.15.0/shadow-4.15.0/lib/readpassphrase.h > ...... > #ifdef LIBBSD_OVERLAY > #include <sys/cdefs.h> > #endif > ...... > ----------------------------------------------------------------------- shadow does not use overlay mode by default, it makes that decision based on other factors in configure.ac: if test "$with_libbsd" != "no"; then AC_SEARCH_LIBS([readpassphrase], [bsd], [], [ AC_MSG_ERROR([readpassphrase() is missing, either from libc or libbsd]) ]) AS_IF([test "$ac_cv_search_readpassphrase" = "-lbsd"], [ PKG_CHECK_MODULES([LIBBSD], [libbsd-overlay]) ]) dnl Make sure either the libc or libbsd provide the header. save_CFLAGS="$CFLAGS" CFLAGS="$CFLAGS $LIBBSD_CFLAGS" AC_CHECK_HEADERS([readpassphrase.h]) AS_IF([test "$ac_cv_header_readpassphrase_h" != "yes"], [ AC_MSG_ERROR([readpassphrase.h is missing]) ]) CFLAGS="$save_CFLAGS" AC_DEFINE(WITH_LIBBSD, 1, [Build shadow with libbsd support]) else AC_DEFINE(WITH_LIBBSD, 0, [Build shadow without libbsd support]) fi AM_CONDITIONAL(WITH_LIBBSD, test x$with_libbsd = xyes) > The same as mg : > ----------------------------------------------------------------------- > $ vim tmp/work/cortexa57-poky-linux/mg/20230501/git/CMakeLists.txt > if(CMAKE_SYSTEM_NAME MATCHES "Linux") > pkg_check_modules (BSD REQUIRED libbsd-overlay) > link_directories (${BSD_LIBRARY_DIRS}) > string (REPLACE ";" " " LIBBSD_FLAGS "${BSD_CFLAGS}") > target_link_libraries (mg ${BSD_LIBRARIES}) > set (CMAKE_C_FLAGS "-Wall -DREGEX -D_GNU_SOURCE ${LIBBSD_FLAGS} ${NCURSES_FLAGS} -L${NCURSES_LIBRARY_DIRS}") > else() > set (CMAKE_C_FLAGS "-Wall -DREGEX ${LIBBSD_FLAGS} ${NCURSES_FLAGS} -L${NCURSES_LIBRARY_DIRS}") > endif() > ----------------------------------------------------------------------- This one does use overlay mode by default, but that doesn't mean every other project does the same. We have to continue supporting both options until libbsd upstream provides only one option. > So, I think the header file after oe_multilib_header should be in overlay mode as the other header files in libbsd. > Of course, If added "#ifdef LIBBSD_OVERLAY" into the header file after oe_multilib_header, libbsd will have a better compatibility. What do you think? I don't think this is correct. Without sed, both modes will work (unless you can demonstrate how it will break; I asked that, and there's no answer in the above). With sed, only overlay mode will work, and now you're proposing additional fixing to undo that effect. So I'd suggest you submit the patch without sed (just oe_multilib parts), and we can then consider if any further fixing is required. Again: please show what is the problem that would occur without sed. Alex
Hi, Alex > So I'd suggest you submit the patch without sed (just oe_multilib parts), and we > can then consider if any further fixing is required. > Again: please show what is the problem that would occur without sed. I' sorry. But my first V1 version patch without sed (just oe_multilib parts) did not pass Khem Raj's compile test (bitbake mg https://errors.yoctoproject.org/Errors/Details/759473/). I'm not sure if it is the problem you want to know. So, I reproduce the build error and found that the cdefs.h that created by multilib_header has the path "#include <bsd/sys/cdefs-32.h>". But mg find herder files without namespaced with bsd/. So, it failed. Shadow has the same build issue. So, I submitted V2 patch with sed. > With sed, only overlay mode will work, and now you're proposing additional > fixing to undo that effect. Yes, I just tested my patch with sed on two recipes (mg and shadow) . The recipes that depend on libbsd are too few to have more test. Best regards Lei > -----Original Message----- > From: Alexander Kanavin <alex.kanavin@gmail.com> > Sent: Friday, April 26, 2024 4:40 PM > To: Lei, Maohui <leimaohui@fujitsu.com> > Cc: openembedded-core@lists.openembedded.org > Subject: Re: [OE-core][PATCH v3] libbsd: Fix conflict error when enable multilib. > > On Fri, 26 Apr 2024 at 08:55, Maohui Lei (Fujitsu) <leimaohui@fujitsu.com> > wrote: > > From the build log and the source code of libbsd, I found that libbsd does > provide header file in overlay mode by default as the man page says. > > ---------------------------------------------------------------------- > > - $ ls > > tmp/work/cortexa57-poky-linux/libbsd/0.12.2/image/usr/lib64/pkgconfig/ > > libbsd-ctor.pc libbsd-overlay.pc libbsd.pc $ grep include > > tmp/work/cortexa57-poky-linux/libbsd/0.12.2/temp/log.do_compile -r > > ...... -I.. -isystem ../../libbsd-0.12.2/include/bsd/...... > > ...... -I.. -isystem ../../libbsd-0.12.2/include/bsd/...... > > ...... > > $ vim > > tmp/work/cortexa57-poky-linux/libbsd/0.12.2/libbsd-0.12.2/include/bsd/ > > stringlist.h > > #ifdef LIBBSD_OVERLAY > > #include <sys/cdefs.h> > > #else > > #include <bsd/sys/cdefs.h> > > #endif > > #include <sys/types.h> > > ---------------------------------------------------------------------- > > The man page does not say overlay mode is default, it only says that it's > recommended. And indeed the above shows that you need to opt into it by > defining LIBBSD_OVERLAY. Let's see how example projects handle it. > > > And I Investigated several OSS(e.g. shadow, mg, libxdmcp) that depend on > libbsd. From the source code of theirs. I found that they does used libbsd in > overlay mode by default. > > Such as shadow: > > ---------------------------------------------------------------------- > > - vim > > tmp/work/cortexa57-poky-linux/shadow/4.15.0/shadow-4.15.0/lib/readpass > > phrase.h > > ...... > > #ifdef LIBBSD_OVERLAY > > #include <sys/cdefs.h> > > #endif > > ...... > > ---------------------------------------------------------------------- > > - > > shadow does not use overlay mode by default, it makes that decision based on > other factors in configure.ac: > > if test "$with_libbsd" != "no"; then > AC_SEARCH_LIBS([readpassphrase], [bsd], [], [ > AC_MSG_ERROR([readpassphrase() is missing, either from > libc or libbsd]) > ]) > AS_IF([test "$ac_cv_search_readpassphrase" = "-lbsd"], [ > PKG_CHECK_MODULES([LIBBSD], [libbsd-overlay]) > ]) > dnl Make sure either the libc or libbsd provide the header. > save_CFLAGS="$CFLAGS" > CFLAGS="$CFLAGS $LIBBSD_CFLAGS" > AC_CHECK_HEADERS([readpassphrase.h]) > AS_IF([test "$ac_cv_header_readpassphrase_h" != "yes"], [ > AC_MSG_ERROR([readpassphrase.h is missing]) > ]) > CFLAGS="$save_CFLAGS" > AC_DEFINE(WITH_LIBBSD, 1, [Build shadow with libbsd support]) else > AC_DEFINE(WITH_LIBBSD, 0, [Build shadow without libbsd support]) fi > AM_CONDITIONAL(WITH_LIBBSD, test x$with_libbsd = xyes) > > > > The same as mg : > > ---------------------------------------------------------------------- > > - $ vim tmp/work/cortexa57-poky-linux/mg/20230501/git/CMakeLists.txt > > if(CMAKE_SYSTEM_NAME MATCHES "Linux") > > pkg_check_modules (BSD REQUIRED libbsd-overlay) > > link_directories (${BSD_LIBRARY_DIRS}) > > string (REPLACE ";" " " LIBBSD_FLAGS "${BSD_CFLAGS}") > > target_link_libraries (mg ${BSD_LIBRARIES}) > > set (CMAKE_C_FLAGS "-Wall -DREGEX -D_GNU_SOURCE ${LIBBSD_FLAGS} > > ${NCURSES_FLAGS} -L${NCURSES_LIBRARY_DIRS}") > > else() > > set (CMAKE_C_FLAGS "-Wall -DREGEX ${LIBBSD_FLAGS} > ${NCURSES_FLAGS} > > -L${NCURSES_LIBRARY_DIRS}") > > endif() > > ---------------------------------------------------------------------- > > - > > This one does use overlay mode by default, but that doesn't mean every other > project does the same. We have to continue supporting both options until libbsd > upstream provides only one option. > > > So, I think the header file after oe_multilib_header should be in overlay mode > as the other header files in libbsd. > > Of course, If added "#ifdef LIBBSD_OVERLAY" into the header file after > oe_multilib_header, libbsd will have a better compatibility. What do you think? > > I don't think this is correct. Without sed, both modes will work (unless you can > demonstrate how it will break; I asked that, and there's no answer in the above). > With sed, only overlay mode will work, and now you're proposing additional > fixing to undo that effect. > > So I'd suggest you submit the patch without sed (just oe_multilib parts), and we > can then consider if any further fixing is required. > Again: please show what is the problem that would occur without sed. > > Alex
On Sun, 28 Apr 2024 at 05:17, Maohui Lei (Fujitsu) <leimaohui@fujitsu.com> wrote: > I' sorry. But my first V1 version patch without sed (just oe_multilib parts) did not pass Khem Raj's compile test (bitbake mg https://errors.yoctoproject.org/Errors/Details/759473/). I'm not sure if it is the problem you want to know. > So, I reproduce the build error and found that the cdefs.h that created by multilib_header has the path "#include <bsd/sys/cdefs-32.h>". > But mg find herder files without namespaced with bsd/. So, it failed. Shadow has the same build issue. > So, I submitted V2 patch with sed. Right, this is the error: | In file included from /srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-mg/20230501/lib64-recipe-sysroot/usr/include/bsd/sys/cdefs.h:27, | from /srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-mg/20230501/lib64-recipe-sysroot/usr/include/bsd/sys/queue.h:39, | from region.c:12: | /srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-mg/20230501/lib64-recipe-sysroot/usr/include/bsd/sys/cdefs-64.h:49:24: error: no include path in which to search for sys/cdefs.h | 49 | #if __has_include_next(<sys/cdefs.h>) | | ^~~~~~~~~~~~~ I'm utterly confused by that. Why can't the compiler find usr/include/sys/cdefs.h? Why is it then happily finding it when bsd/ prefix is dropped from cdefs-64 include statement in usr/include/bsd/sys/cdefs.h ? But the fix does seem to help, and makes multilib selection header match libbsd headers (in overlay mode). So: - the patch is ok; but please add a note (a comment in the recipe just above sed) that it hasn't been tested when libbsd is used in 'namespaced headers' mode according to https://man.archlinux.org/man/libbsd.7 Alex
Hi, Alex > - the patch is ok; but please add a note (a comment in the recipe just above > sed) that it hasn't been tested when libbsd is used in 'namespaced headers' > mode according to > https://man.archlinux.org/man/libbsd.7 OK, I will add this note into recipe too and send V4 patch. Best regards Lei > -----Original Message----- > From: Alexander Kanavin <alex.kanavin@gmail.com> > Sent: Monday, April 29, 2024 6:46 PM > To: Lei, Maohui <leimaohui@fujitsu.com> > Cc: openembedded-core@lists.openembedded.org > Subject: Re: [OE-core][PATCH v3] libbsd: Fix conflict error when enable > multilib. > > On Sun, 28 Apr 2024 at 05:17, Maohui Lei (Fujitsu) <leimaohui@fujitsu.com> > wrote: > > I' sorry. But my first V1 version patch without sed (just oe_multilib parts) > did not pass Khem Raj's compile test (bitbake mg > https://errors.yoctoproject.org/Errors/Details/759473/). I'm not sure if it is > the problem you want to know. > > So, I reproduce the build error and found that the cdefs.h that created by > multilib_header has the path "#include <bsd/sys/cdefs-32.h>". > > But mg find herder files without namespaced with bsd/. So, it failed. > Shadow has the same build issue. > > So, I submitted V2 patch with sed. > > Right, this is the error: > > | In file included from > /srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/ > lib64-mg/20230501/lib64-recipe-sysroot/usr/include/bsd/sys/cdefs.h:27, > | from > /srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/ > lib64-mg/20230501/lib64-recipe-sysroot/usr/include/bsd/sys/queue.h:39, > | from region.c:12: > | > /srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/ > lib64-mg/20230501/lib64-recipe-sysroot/usr/include/bsd/sys/cdefs-64.h:49: > 24: > error: no include path in which to search for sys/cdefs.h > | 49 | #if __has_include_next(<sys/cdefs.h>) > | | ^~~~~~~~~~~~~ > > I'm utterly confused by that. Why can't the compiler find > usr/include/sys/cdefs.h? > Why is it then happily finding it when bsd/ prefix is dropped from > cdefs-64 include statement in usr/include/bsd/sys/cdefs.h ? > > But the fix does seem to help, and makes multilib selection header match > libbsd headers (in overlay mode). > > So: > - the patch is ok; but please add a note (a comment in the recipe just above > sed) that it hasn't been tested when libbsd is used in 'namespaced headers' > mode according to > https://man.archlinux.org/man/libbsd.7 > > Alex
diff --git a/meta/recipes-support/libbsd/libbsd_0.12.2.bb b/meta/recipes-support/libbsd/libbsd_0.12.2.bb index 7d5e88f293..e1fc7d9b3c 100644 --- a/meta/recipes-support/libbsd/libbsd_0.12.2.bb +++ b/meta/recipes-support/libbsd/libbsd_0.12.2.bb @@ -40,8 +40,14 @@ SRC_URI = "https://libbsd.freedesktop.org/releases/${BPN}-${PV}.tar.xz" SRC_URI[sha256sum] = "b88cc9163d0c652aaf39a99991d974ddba1c3a9711db8f1b5838af2a14731014" -inherit autotools pkgconfig +inherit autotools pkgconfig multilib_header DEPENDS += "libmd" +do_install:append () { + oe_multilib_header bsd/sys/cdefs.h + sed -i 's:#include <bsd/sys/cdefs:#include <sys/cdefs:g' ${D}${includedir}/bsd/sys/cdefs.h +} + + BBCLASSEXTEND = "native nativesdk"