Search code examples
linuxgithubpackagepatchbuildroot

Buildroot - Github package re-extracted at every build

I'm currently trying to integrate a package from github which implements a MTP-responder in userspace.

I'm using buildroot 2017.08.

I have created both Config.in and umtprd.mk files with the following content.

I also patch sources in order to replace some fields' value into the configuration file given with the package.

Config.in

$ cat Config.in
config BR2_PACKAGE_UMTPRD
    bool "umtprd"
    help
      umtprd is a deamon, checking USB connection and start MTP communication.

      https://github.com/viveris/uMTP-Responder

if BR2_PACKAGE_UMTPRD
config BR2_PACKAGE_UMTPRD_TAG
    string "git version of umtprd package"
    default "master"
    help
      Must be the name of the branch/tag :
      https://github.com/viveris/uMTP-Responder

endif #BR2_PACKAGE_UMTPRD

umtprd.mk

$ cat umtprd.mk
UMTPRD_VERSION = $(BR2_PACKAGE_UMTPRD_TAG)
UMTPRD_SITE_METHOD = git
UMTPRD_SITE = https://github.com/viveris/uMTP-Responder

define UMTPRD_BUILD_CMDS
    $(MAKE) CFLAGS="-DUSE_SYSLOG" CC="$(TARGET_CC)" LD="$(TARGET_LD)" -C $(@D)
endef

define UMTPRD_INSTALL_TARGET_CMDS
    $(INSTALL) -D -m 0755 $(@D)/umtprd $(TARGET_DIR)/usr/bin
    $(INSTALL) -D -m 0755 $(@D)/conf/umtprd*.sh $(TARGET_DIR)/usr/bin
    $(INSTALL) -D -m 0755 $(@D)/conf/S98uMTPrd $(TARGET_DIR)/etc/init.d
    mkdir -p $(TARGET_DIR)/etc/umtprd
    $(INSTALL) -D -m 0644 $(@D)/conf/umtprd.conf $(TARGET_DIR)/etc/umtprd/
endef

$(eval $(generic-package))

Patch

$ cat 0001-configuration.patch 
diff -Naur a/conf/umtprd.conf b/conf/umtprd.conf
--- a/conf/umtprd.conf  2018-12-22 14:58:25.000000000 +0100
+++ b/conf/umtprd.conf  2019-01-10 10:31:06.069769073 +0100
@@ -11,8 +11,7 @@
 #storage command : Create add a storage entry point. Up to 16 entry points supported
 #Syntax : storage "PATH" "NAME"

-storage "/"      "root folder"
-storage "/home"  "home folder"
+storage "<path>" "<name>"

 # Set the USB manufacturer string

@@ -20,11 +19,11 @@

 # Set the USB Product string

-product "The Viveris Product !"
+product "<product_string>"

 # Set the USB Serial number string

-serial "01234567"
+serial "<serial>"

 # Set the USB interface string. Should be always "MTP"

First build

When I first execute make, the package is successfully downloaded, extracted, patched, built and installed.

$ make
>>> umtprd umtprd-0.9.7 Downloading
[...]
warning: refname 'umtprd-0.9.7' is ambiguous.
WARNING: no hash file for umtprd-umtprd-0.9.7.tar.gz
>>> umtprd umtprd-0.9.7 Extracting
[...]
>>> umtprd umtprd-0.9.7 Patching

Applying 0001-configuration.patch using patch: 
patching file conf/umtprd.conf
>>> umtprd umtprd-0.9.7 Configuring
>>> umtprd umtprd-0.9.7 Building
[...]
>>> umtprd umtprd-0.9.7 Installing to target
[...]
>>>   Finalizing target directory
[...]
>>>   Sanitizing RPATH in target tree
[...]
>>>   Copying overlay <path_to_overlay>
>>>   Executing post-build script <post_build_script>
>>>   Generating root filesystem image rootfs.tar
[...]
>>>   Executing post-image script <post_image_script>

Later build

However if I run make again without removing the folder output/build/umtprd-umtprd-0.9.7 buildroot tries to extract and patch the sources again.

In that case the patch obviously fails, thus the compiling process is stopped and images are not generated.

$ make
WARNING: no hash file for umtprd-umtprd-0.9.7.tar.gz
>>> umtprd umtprd-0.9.7 Extracting
[...]
>>> umtprd umtprd-0.9.7 Patching

Applying 0001-configuration.patch using patch: 
Error: duplicate filename '0001-configuration.patch'
Conflicting files are:
  already applied: <path_to_patch>/0001-configuration.patch
  to be applied  : <path_to_patch>/0001-configuration.patch
package/pkg-generic.mk:191 : la recette pour la cible « <path_to_buildroot>/output/build/umtprd-"umtprd-0.9.7"/.stamp_patched » a échouée
make: *** [<path_to_buildroot>/output/build/umtprd-"umtprd-0.9.7"/.stamp_patched] Erreur 1

I have check inside the build directory and all .stamp files seems to be there so I don't understand why buildroot re-extract, re-patch and re-build the package since configuration and sources haven't changed.

$ ls -la output/build/umtprd-umtprd-0.9.7/.stamp_*
-rw-r--r-- 1 <user> <user> 0 janv. 10 14:58 output/build/umtprd-umtprd-0.9.7/.stamp_built
-rw-r--r-- 1 <user> <user> 0 janv. 10 14:58 output/build/umtprd-umtprd-0.9.7/.stamp_configured
-rw-r--r-- 1 <user> <user> 0 janv. 10 15:05 output/build/umtprd-umtprd-0.9.7/.stamp_downloaded
-rw-r--r-- 1 <user> <user> 0 janv. 10 15:05 output/build/umtprd-umtprd-0.9.7/.stamp_extracted
-rw-r--r-- 1 <user> <user> 0 janv. 10 14:58 output/build/umtprd-umtprd-0.9.7/.stamp_patched
-rw-r--r-- 1 <user> <user> 0 janv. 10 14:58 output/build/umtprd-umtprd-0.9.7/.stamp_target_installed

The problem can be solved by removing the build folder output/build/umtprd-umtprd-0.9.7 but I shouldn't have to do it, thus I would like to understand why buildroot is re-extracting and re-patching sources.

What did I do wrong here ?

Solution

  • You should change the following:

    UMTPRD_VERSION = $(call qstrip,$(BR2_PACKAGE_UMTPRD_TAG))

    The Kconfig variable BR2_PACKAGE_UMTPRD_TAG contains quotes, its value is e.g. "master" including the quotes. For make, quotes are nothing special. So in the dependency chain, it will look for a stamp file output/build/umtprd-"master"/.stamp_extracted. However, when the stamp file is created, it goes through the shell which removes the quotes, so the actual file created is output/build/umtprd-master. Thus, make thinks that the stamp file doesn't exist yet.

    Note that it is not just the extraction that is repeated, also the download. However, in the download step there is an additional check that skips the download if the file-to-be-downloaded exists already.