"Build Your Application" (Quick Start section 2.7) doesn't work

[bsterling]

I followed the Quick Start instructions verbatim. When I got to 2.7, the build fails. This failure as well as the suggested workaround is documented in ​https://lists.rtems.org/pipermail/users/2023-July/068846.html. I'm repeating it here in order to avoid cross-references.

I'm filing this against the documentation rather than the waf tool simply because the waf tool may be functioning as desired, but the documentation needs to be updated.

Ok, in order to eliminate the concerns that I had specified directories or parameter incorrectly, I restarted the 'Quick Start' (​https://docs.rtems.org/branches/master/user/start/index.html) from scratch. The application build still fails (with the errors I saw before).

Below are all of the steps I did (I followed exactly​ along with the 'Quick Start' guide - you are welcome to verify everything I did). The errors (which are the same errors that my co-worker got) that occur with the application configure (​https://docs.rtems.org/branches/master/user/start/app.html) are shown after all of the commands.
One thing I noticed was that there's a difference in the 'waf' options between the BSP configure and the application configure. However, when I changed '--rtems=' to '--prefix=' in the failing case, the command still failed.
Given that I followed the 'Quick Start' explicitly, I still think there's something wrong with the 'application' part of the 'quick start' (as verified by another co-worker). As such, I'll try anything that can be suggested.

I'm also willing to help debug the 'non-git clone' mechanisms if that's where people want to start. In the meantime, things are going a little bit slow with trying to use RTEMS :-) .

Thanks in advance.

(NOTE - I had to use the git clone mechanism because the other mechanisms were not working)
mkdir -p $HOME/quick-start/src
cd $HOME/quick-start/src
git clone ​git://git.rtems.org/rtems-source-builder.git rsb
git clone ​git://git.rtems.org/rtems.git

cd $HOME/quick-start/src/rsb/rtems
../source-builder/sb-set-builder --prefix=$HOME/quick-start/rtems/6 6/rtems-sparc

(NOTE - I had to manually build because the 'RSB BSP Build' did not work)
mkdir -p $HOME/quick-start/build/b-erc32
export PATH=$HOME/quick-start/rtems/6/bin:"$PATH"

command -v sparc-rtems6-gcc && echo "found"

echo "not found"

cd $HOME/quick-start/src/rtems
echo "[sparc/erc32]" > config.ini
echo "BUILD_TESTS = True" >> config.ini
./waf configure --prefix=$HOME/quick-start/rtems/6
cd $HOME/quick-start/src/rtems
./waf
cd $HOME/quick-start/src/rtems
./waf install

cd $HOME/quick-start/src/rtems
rtems-test --rtems-bsp=erc32-sis build/sparc/erc32

mkdir -p $HOME/quick-start/app/hello
cd $HOME/quick-start/app/hello
curl ​https://waf.io/waf-2.0.19 > waf
chmod +x waf
git init
git submodule add ​git://git.rtems.org/rtems_waf.git rtems_waf
(Create the 3 specified application files)
./waf configure --rtems=$HOME/quick-start/rtems/6 --rtems-bsp=sparc/erc32

The errors I get are:

~/quick-start/app/hello$ ./waf configure --rtems=$HOME/quick-start/rtems/6 --rtems-bsp=sparc/erc32Setting top to : /home/bsterling/quick-start/app/hello
Setting out to : /home/bsterling/quick-start/app/hello/build
Traceback (most recent call last):

File "/home/bsterling/quick-start/app/hello/.waf3-2.0.19-1f3c580272b15a03d2566843c5fe872a/waflib/Scripting.py", line 119, in waf_entry_point

run_commands()

File "/home/bsterling/quick-start/app/hello/.waf3-2.0.19-1f3c580272b15a03d2566843c5fe872a/waflib/Scripting.py", line 182, in run_commands

ctx=run_command(cmd_name)

File "/home/bsterling/quick-start/app/hello/.waf3-2.0.19-1f3c580272b15a03d2566843c5fe872a/waflib/Scripting.py", line 173, in run_command

ctx.execute()

File "/home/bsterling/quick-start/app/hello/.waf3-2.0.19-1f3c580272b15a03d2566843c5fe872a/waflib/Configure.py", line 85, in execute

super(ConfigurationContext?,self).execute()

File "/home/bsterling/quick-start/app/hello/.waf3-2.0.19-1f3c580272b15a03d2566843c5fe872a/waflib/Context.py", line 92, in execute

self.recurse([os.path.dirname(g_module.root_path)])

File "/home/bsterling/quick-start/app/hello/.waf3-2.0.19-1f3c580272b15a03d2566843c5fe872a/waflib/Context.py", line 133, in recurse

user_function(self)

File "/home/bsterling/quick-start/app/hello/wscript", line 26, in configure

rtems.configure(conf, bsp_configure = bsp_configure)

File "/home/bsterling/quick-start/app/hello/rtems_waf/rtems.py", line 190, in configure

check_options(conf,

File "/home/bsterling/quick-start/app/hello/rtems_waf/rtems.py", line 496, in check_options

arch_bsps = _check_arch_bsps(rtems_bsps, rtems_config, rtems_path,

File "/home/bsterling/quick-start/app/hello/rtems_waf/rtems.py", line 851, in _check_arch_bsps

installed = _find_installed_arch_bsps(config, path, archs, version)

File "/home/bsterling/quick-start/app/hello/rtems_waf/rtems.py", line 823, in _find_installed_arch_bsps

if _arch_from_arch_bsp(f[:-3]) in archs:

File "/home/bsterling/quick-start/app/hello/rtems_waf/rtems.py", line 862, in _arch_from_arch_bsp

rtems_field_index = next(i for i, field in enumerate(fields) if field.startswith('rtems'))

StopIteration?

[John Willis]

I had this same issue. I traced the problem to the fact that line 862 mentioned in the most recent call in the traceback in the ticket has a bug in the call next(i for i, field in enumerate(fields) if field.startswith('rtems))

This causes a stop iteration when the generator, on which we are calling next, is empty. In my case, the directory being searched for files ending in .pc and starting with rtems consists of :

./rtems/6/lib/pkgconfig/sparc-rtems6-erc32.pc
./rtems/6/lib/pkgconfig/gmp.pc
./rtems/6/lib/pkgconfig/expat.pc
./rtems/6/lib/pkgconfig/sparc-rtems6-leon3.pc

when the function gets to "gmp.pc" or "expat.pc", it will fail with a StopIteration? because we are calling next on an empty generator.

A proposed fix here is to just wrap the call to next on the generator with a try/catch, and return None if we catch a StopIteration?

[Chris Johns]

John, I have lost track of this issue, is there a patch I need to push I have missed?

[John Willis]

Replying to Chris Johns:

John, I have lost track of this issue, is there a patch I need to push I have missed?

Hi Chris. Thanks for looping back around on this.

It looks like this bug is still on master as of 2023-08-10. My comment has a proposed patch for this, that I have implemented on a fork. We could also just not use a generator for this particular function -- the largest the generator could possibly be is size 1, so seems a bit excessive as opposed to using a list comprehension or some if statements. That being said, I do appreciate the compact (functional) nature of the use of the generator, but if going that direction we need to catch the StopIteration? in the event that it is empty. Also, adding some unit tests here would be really nice.

I am happy to provide more info -- stack traces, examples, etc -- to help, if needed/desired.

[Hamilton Henneberg]

I am currently having the same issues with the current rtems_waf build, as of 2023-09-19. I was wondering if a patch was ever pushed to the current build, and if not, was a separate solution ever found?

[John Willis]

Replying to Hamilton Henneberg:

I am currently having the same issues with the current rtems_waf build, as of 2023-09-19. I was wondering if a patch was ever pushed to the current build, and if not, was a separate solution ever found?

Hi Hamilton. I found a solution to it and am waiting on Chris to get back to us about it. I have a fork of rtems_waf that patches this by handling the StopIteration? when the generator being called "next" on is empty. I outline the solution for it above in my first comment:

A proposed fix here is to just wrap the call to next on the generator with a try/catch, and return None if we catch a StopIteration

[Chris Johns]

I am sorry if I have dropped the ball on this.

Is there a patch for this I can push with the appropriate attributes etc?

[John Willis]

Replying to Chris Johns:

I am sorry if I have dropped the ball on this.

Is there a patch for this I can push with the appropriate attributes etc?

Hey Chris. I have a patch. I am having a smtp failure though when trying to do git send-email though unfortunately.

[Chris Johns]

Replying to John Willis:

Replying to Chris Johns:

I am sorry if I have dropped the ball on this.

Is there a patch for this I can push with the appropriate attributes etc?

Hey Chris. I have a patch. I am having a smtp failure though when trying to do git send-email though unfortunately.

We have this ​https://docs.rtems.org/branches/master/eng/vc-users.html. All we need is your details for the change so it is attributed to you.

[John Willis]

Replying to Chris Johns:

Replying to John Willis:

Replying to Chris Johns:

I am sorry if I have dropped the ball on this.

Is there a patch for this I can push with the appropriate attributes etc?

Hey Chris. I have a patch. I am having a smtp failure though when trying to do git send-email though unfortunately.

We have this ​https://docs.rtems.org/branches/master/eng/vc-users.html. All we need is your details for the change so it is attributed to you.

I had looked through those docs and created a patch, but like I said I am unable to email due to smtp error -- this is probably due to Google's updated policies on third party apps in recent years. I am happy to email it directly, as well, but for the time being here is the patch:

From da95beece91ee5d4ea820969804250d661797935 Mon Sep 17 00:00:00 2001
From: jmart <jmart.will@gmail.com>
Date: Wed, 4 Oct 2023 15:47:44 -0600
Subject: [PATCH] [PATCH] wrapped call to _arch_from_arch_bsp in
 _find_insalled_arch_bsps with a try/except to catch a possible StopIteration.
 Ignore the exception if it is caught. UPDATE #4927

---
 rtems.py | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/rtems.py b/rtems.py
index c65a7d2..2be0261 100644
--- a/rtems.py
+++ b/rtems.py
@@ -820,8 +820,11 @@ def _find_installed_arch_bsps(config, path, archs, version):
     if config is None:
         for f in os.listdir(_pkgconfig_path(path)):
             if f.endswith('.pc'):
-                if _arch_from_arch_bsp(f[:-3]) in archs:
-                    arch_bsps += [f[:-3]]
+                try:
+                    if _arch_from_arch_bsp(f[:-3]) in archs:
+                        arch_bsps += [f[:-3]]
+                except StopIteration:
+                    pass
     else:
         ab = subprocess.check_output([config, '--list-format'])
         ab = ab[:-1].replace('"', '')
-- 
2.34.1

[Chris Johns]

Can this now be closed given Martin's patch was pushed?

[John Willis]

Replying to Chris Johns:

Can this now be closed given Martin's patch was pushed?

I found Martin's commit. I am worried that this will cause other problems. This is a very similar solution that I initially had, but the following lines in configure in rtems.py worry me:

arch = _arch_from_arch_bsp(ab)
bsp = _bsp_from_arch_bsp(ab)

conf.env.ARCH_BSP = '%s/%s' % (arch.split('-')[0], bsp)

If we return None from _arch_from_arch_bsp then arch.split('-')[0] will fail.

[Chris Johns]

Replying to John Willis:

Replying to Chris Johns:

Can this now be closed given Martin's patch was pushed?

I found Martin's commit. I am worried that this will cause other problems. This is a very similar solution that I initially had, but the following lines in configure in rtems.py worry me:

arch = _arch_from_arch_bsp(ab)
bsp = _bsp_from_arch_bsp(ab)

conf.env.ARCH_BSP = '%s/%s' % (arch.split('-')[0], bsp)

If we return None from _arch_from_arch_bsp then arch.split('-')[0] will fail.

Do you have a suggestion?

[John Willis]

Replying to Chris Johns:

Replying to John Willis:

Replying to Chris Johns:

Can this now be closed given Martin's patch was pushed?

I found Martin's commit. I am worried that this will cause other problems. This is a very similar solution that I initially had, but the following lines in configure in rtems.py worry me:

arch = _arch_from_arch_bsp(ab)
bsp = _bsp_from_arch_bsp(ab)

conf.env.ARCH_BSP = '%s/%s' % (arch.split('-')[0], bsp)

If we return None from _arch_from_arch_bsp then arch.split('-')[0] will fail.

Do you have a suggestion?

This might be overkill, but my thought here is to have the caller handle the case that the return value is None, and to fail elegantly as opposed to opaquely:

From c22c450f2fa39e25ea2c3f789df5df12e59e7c3f Mon Sep 17 00:00:00 2001
From: jmart <jmart.will@gmail.com>
Date: Sat, 7 Oct 2023 11:33:10 -0600
Subject: [PATCH] If we try to call _arch_from_arch_bsp in config on an
 arch_bsp that does not contain the string 'rtems', then we will return None,
 and eventually try to call split('-') on none type, which will fail. Instead,
 we could catch this situation and raise an informative exception to the user.

---
 rtems.py | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/rtems.py b/rtems.py
index 1566e58..37cc32e 100644
--- a/rtems.py
+++ b/rtems.py
@@ -222,7 +222,12 @@ def configure(conf, bsp_configure=None):
         conf.msg('Long commands', long_commands)
 
         arch = _arch_from_arch_bsp(ab)
+        if arch is None:
+            raise Exception(f"No valid arch in {ab}")
+        
         bsp = _bsp_from_arch_bsp(ab)
+        if bsp is None:
+            raise Exception(f"No valid bsp in {ab}")
 
         conf.env.ARCH_BSP = '%s/%s' % (arch.split('-')[0], bsp)
 
-- 
2.34.1

This is definitely an edge case and given that there are no tests in this repo, it is hard too say if this will ever occur -- it seems like this situation would be caught earlier on in the process since we would be trying to build an unsupported bsp. In any case, if we don't do something like this, I think I am good with Martin's patch.

Thanks for your patience.