# Release notes of RTEMS SMP QDP release 4 Document generated at 2022-06-15 ## SPRs solved ### 781: Traceability from SRS to SVS/SVR incomplete - Created at: 2022-01-31T14:24:51.378Z - Update at: 2022-04-13T16:00:21.018Z Description: There is a problem with the traceability from SRS to SVS/SVR. Not all validation items listed in the SRS can be found in the SVS/SVR. The trace is correct for requirements that are covered by a test case. However, as it turns out requirements could also be validated by a test suite and in those cases the trace is not reported in table 1, section validation requirements of the SRS. This needs to be implemented. ## SPRs open ### 766: QR2-JFS-10 The QDP generation contains some error message - Created at: 2021-10-15T07:46:32.496Z - Update at: 2022-04-13T14:32:17.123Z Description: The latest QDP generations contains error messages. The log is attached to the RID. The generation process doesn't show error. Please clarify the errors and fix them.[log_generate_qdps.log](/uploads/b09e350189b3bdba071251c31d8d819d/log_generate_qdps.log) ### 741: QR2 SVS and SUITP Generation Issues - Created at: 2021-09-29T13:08:52.043Z - Update at: 2022-04-13T14:28:50.483Z Description: This list contains issues with the SVS and SUITP document generation: * [ ] Glossary terms used by the test specification are missing. I guess this is due to a problem with an item mapper or the glossary generator. * [ ] There are no links to the tested interfaces and requirements. * [x] There are no document internal links from test suites to test cases and vice versa. * [ ] There are no links to the test code in the SDD. The include test code fragments are not really useful. ### 740: QR2 SRS Generation Issues - Created at: 2021-09-28T11:32:47.813Z - Update at: 2022-04-13T14:39:23.267Z Description: For issues with the SRS specification itself use #739. This list contains issues with the SRS document generation: * [ ] There is no hierarchical overview of the requirements, see for example: `5.2.1 Specification Item Hierarchy` in the RTEMS Software Engineering Manual. * [x] Parent links with the role ``function-implementation`` are not included, see ``specview.py``. * [ ] Glossary terms used by the specification are missing. I guess this is due to a problem with an item mapper or the glossary generator. * [ ] There are no document external links in some description texts. For example: spec:/bsp/sparc/req/grlib-load-08 * [x] Items of type `requirement/non-functional/design-group` are missing, for example: spec:/newlib/req/futex. * [x] There are no links to interface and validation items. * [ ] The chapter `Validation Requirements` should not only print a validation method for each requirement item which needs validation. It should also present/link the path of items which validate an interface. Some requirements may be validated by several validation items. ### 734: QR2 ICD Generation Issues (Was Finding: SVR: ) - Created at: 2021-09-15T14:32:41.268Z - Update at: 2022-04-13T14:29:36.526Z Description: For issues with the ICD specification itself use #735. This list contains issues with the ICD document generation: * [x] Links to some sections are invalid. Example: `This interface container belongs to the SectionUseOfInterface interface domain.` of `spec://user/if/container`. There are lots of `WARNING: undefined label: sectiondocumentation` warnings in the QDP build log. * [x] Referenced items in interface domain membership statements are wrong. Example `This interface unspecified belongs to the spec:/user/if/container interface domain.` of `spec:/user/if/asm`. The referenced interface domain should be `spec:/user/if/domain`. * [ ] Glossary terms used by the interface specification are missing. I guess this is due to a problem with an item mapper or the glossary generator. * [ ] The presentation of application configuration options is incomplete. Example: Compare ICD entry for `spec:/acfg/if/appl-needs-clock-driver` with https://docs.rtems.org/branches/master/c-user/config/device-driver.html#configure-application-needs-clock-driver * [ ] The presentation of functions should be aligned with the Classic API Guide. Example: Compare ICD entry for `spec:/rtems/task/if/construct` with https://docs.rtems.org/branches/master/c-user/task/directives.html#rtems-task-construct * [ ] The formatting of code blocks should be shared with `interfacedoc.py`. Maybe moved to a separate module. * [ ] There are no document internal links in description texts. Example: ICD entry for `spec:/rtems/task/if/construct` * [ ] There are no links to requirements and validation items. This makes it hard to navigate through the specification layers. * [x] The `interface/register-block` items do not show up in the ICD. They should at least show up with some basic information. * [x] The external references of unspecified interfaces are not present on the ICD. Example: `spec:/c/if/clock-nanosleep`: ``` references: url: https://pubs.opengroup.org/onlinepubs/9699919799/functions/clock_nanosleep.html ``` * [x] What do warnings like this mean: `:Specification /rtems/timer/if/group missing mandatory ingroup or refinement parent link`? The item has an `interface-ingroup` link. * [x] The chapter `Validation Requirements` should not only print a validation method for each interface item which needs validation. It should also present/link the path of items which validate an interface. Some interfaces may be validated by several validation items. * [ ] There is no hierarchical overview of the requirements, see for example: `5.2.1 Specification Item Hierarchy` in the RTEMS Software Engineering Manual. * [x] Parent links with the role ``function-implementation`` are not included, see ``specview.py``. In general, using definition lists could lead to trouble if they are nested. For the Classic API we use now `.. rubric:: ABC`. -------- ~~This is a finding in the SVR Release 4 (Post-QR#2)~~ #### SVR Section *4.5.2.2 Verification of Software Documentation* ##### Point *1. Is the documentation adequate, complete, and consistent?* **Finding:** The following documents are not complete: * ~~Interface Control Document (ICD)~~ Note: This issue is successor of #670 , #737. Review 2021-11-26 closed this finding. It remains open as normal issue to track the above point. ### 673: Task 1: Return status of run_command() needs to be checked - Created at: 2021-03-01T14:39:59.150Z - Update at: 2022-04-13T14:33:27.026Z Description: The gcovr tool I used was outdated, so the HTML code coverage information was missing. However, I got still got a QDP without any obvious error messages. The return status of all run_command() calls should be checked. I suggest to check the process status in run_command() and throw an exception if it is not 0. ### 601: Task 1: The test suite contains too much copy and paste - Created at: 2020-10-30T15:06:12.039Z - Update at: 2022-04-13T14:24:56.073Z Description: The build step tests should use the set_up_package(): ``` from rtemsspec.tests.packageutil import set_up_package ``` Affected tests: ``` rtemsspec/tests/test_svsmanager.py: os.path.join(os.path.dirname(__file__), "../../spec-spec") rtemsspec/tests/test_svsmanager.py: os.path.join(os.path.dirname(__file__), "../../spec-spec") rtemsspec/tests/test_bspbuilder.py: os.path.join(os.path.dirname(__file__), "../../spec-spec"), rtemsspec/tests/test_repoclone.py: os.path.join(os.path.dirname(__file__), "../../spec-spec") rtemsspec/tests/test_sddmanager.py: os.path.join(os.path.dirname(__file__), "../../spec-spec") rtemsspec/tests/test_hashutils.py: os.path.join(os.path.dirname(__file__), "../../spec-spec") rtemsspec/tests/test_documentbuilder.py: os.path.join(os.path.dirname(__file__), "../../spec-spec") rtemsspec/tests/test_documentbuilder.py: os.path.join(os.path.dirname(__file__), "../../spec-spec") rtemsspec/tests/test_rtemsdocbuilder.py: os.path.join(os.path.dirname(__file__), "../../spec-spec"), rtemsspec/tests/test_filecopymanager.py: os.path.join(os.path.dirname(__file__), "../../spec-spec"), rtemsspec/tests/test_prequalifiedonly.py: os.path.join(os.path.dirname(__file__), "../../spec-spec"), rtemsspec/tests/test_prequalifiedonly.py: os.path.join(os.path.dirname(__file__), "../../spec-spec"), rtemsspec/tests/test_rsbbuilder.py: os.path.join(os.path.dirname(__file__), "../../spec-spec"), rtemsspec/tests/test_packagebuild.py: os.path.join(os.path.dirname(__file__), "../../spec-spec") ``` The directory `rtemsspec/tests/config-packagebuild/spec` which contains a duplicated configuration specification must go away. ### 591: Task 1: Copyright handling of generated documents is incomplete - Created at: 2020-10-21T13:16:29.241Z - Update at: 2022-04-13T14:25:46.885Z Description: The copyright information and license should be derived from the items which contribute to the document and the header information of document source files. ### 587: Task 1: Excessive use of subprocesses - Created at: 2020-10-21T07:05:35.431Z - Update at: 2022-04-13T14:26:37.578Z Description: The latest QDP generations contains error messages. The log is attached to the RID. The generation process doesn't show error. Please clarify the errors and fix them.[log_generate_qdps.log](/uploads/b09e350189b3bdba071251c31d8d819d/log_generate_qdps.log) This list contains issues with the SVS and SUITP document generation: * [ ] Glossary terms used by the test specification are missing. I guess this is due to a problem with an item mapper or the glossary generator. * [ ] There are no links to the tested interfaces and requirements. * [x] There are no document internal links from test suites to test cases and vice versa. * [ ] There are no links to the test code in the SDD. The include test code fragments are not really useful. For issues with the SRS specification itself use ###739. This list contains issues with the SRS document generation: * [ ] There is no hierarchical overview of the requirements, see for example: `5.2.1 Specification Item Hierarchy` in the RTEMS Software Engineering Manual. * [x] Parent links with the role ``function-implementation`` are not included, see ``specview.py``. * [ ] Glossary terms used by the specification are missing. I guess this is due to a problem with an item mapper or the glossary generator. * [ ] There are no document external links in some description texts. For example: spec:/bsp/sparc/req/grlib-load-08 * [x] Items of type `requirement/non-functional/design-group` are missing, for example: spec:/newlib/req/futex. * [x] There are no links to interface and validation items. * [ ] The chapter `Validation Requirements` should not only print a validation method for each requirement item which needs validation. It should also present/link the path of items which validate an interface. Some requirements may be validated by several validation items. For issues with the ICD specification itself use ###735. This list contains issues with the ICD document generation: * [x] Links to some sections are invalid. Example: `This interface container belongs to the SectionUseOfInterface interface domain.` of `spec://user/if/container`. There are lots of `WARNING: undefined label: sectiondocumentation` warnings in the QDP build log. * [x] Referenced items in interface domain membership statements are wrong. Example `This interface unspecified belongs to the spec:/user/if/container interface domain.` of `spec:/user/if/asm`. The referenced interface domain should be `spec:/user/if/domain`. * [ ] Glossary terms used by the interface specification are missing. I guess this is due to a problem with an item mapper or the glossary generator. * [ ] The presentation of application configuration options is incomplete. Example: Compare ICD entry for `spec:/acfg/if/appl-needs-clock-driver` with https://docs.rtems.org/branches/master/c-user/config/device-driver.html#configure-application-needs-clock-driver * [ ] The presentation of functions should be aligned with the Classic API Guide. Example: Compare ICD entry for `spec:/rtems/task/if/construct` with https://docs.rtems.org/branches/master/c-user/task/directives.html#rtems-task-construct * [ ] The formatting of code blocks should be shared with `interfacedoc.py`. Maybe moved to a separate module. * [ ] There are no document internal links in description texts. Example: ICD entry for `spec:/rtems/task/if/construct` * [ ] There are no links to requirements and validation items. This makes it hard to navigate through the specification layers. * [x] The `interface/register-block` items do not show up in the ICD. They should at least show up with some basic information. * [x] The external references of unspecified interfaces are not present on the ICD. Example: `spec:/c/if/clock-nanosleep`: ``` references: url: https://pubs.opengroup.org/onlinepubs/9699919799/functions/clock_nanosleep.html ``` * [x] What do warnings like this mean: `:Specification /rtems/timer/if/group missing mandatory ingroup or refinement parent link`? The item has an `interface-ingroup` link. * [x] The chapter `Validation Requirements` should not only print a validation method for each interface item which needs validation. It should also present/link the path of items which validate an interface. Some interfaces may be validated by several validation items. * [ ] There is no hierarchical overview of the requirements, see for example: `5.2.1 Specification Item Hierarchy` in the RTEMS Software Engineering Manual. * [x] Parent links with the role ``function-implementation`` are not included, see ``specview.py``. In general, using definition lists could lead to trouble if they are nested. For the Classic API we use now `.. rubric:: ABC`. -------- ~~This is a finding in the SVR Release 4 (Post-QR#2)~~ #### SVR Section *4.5.2.2 Verification of Software Documentation* ##### Point *1. Is the documentation adequate, complete, and consistent?* **Finding:** The following documents are not complete: * ~~Interface Control Document (ICD)~~ Note: This issue is successor of ###670 , ###737. Review 2021-11-26 closed this finding. It remains open as normal issue to track the above point. The gcovr tool I used was outdated, so the HTML code coverage information was missing. However, I got still got a QDP without any obvious error messages. The return status of all run_command() calls should be checked. I suggest to check the process status in run_command() and throw an exception if it is not 0. The build step tests should use the set_up_package(): ``` from rtemsspec.tests.packageutil import set_up_package ``` Affected tests: ``` rtemsspec/tests/test_svsmanager.py: os.path.join(os.path.dirname(__file__), "../../spec-spec") rtemsspec/tests/test_svsmanager.py: os.path.join(os.path.dirname(__file__), "../../spec-spec") rtemsspec/tests/test_bspbuilder.py: os.path.join(os.path.dirname(__file__), "../../spec-spec"), rtemsspec/tests/test_repoclone.py: os.path.join(os.path.dirname(__file__), "../../spec-spec") rtemsspec/tests/test_sddmanager.py: os.path.join(os.path.dirname(__file__), "../../spec-spec") rtemsspec/tests/test_hashutils.py: os.path.join(os.path.dirname(__file__), "../../spec-spec") rtemsspec/tests/test_documentbuilder.py: os.path.join(os.path.dirname(__file__), "../../spec-spec") rtemsspec/tests/test_documentbuilder.py: os.path.join(os.path.dirname(__file__), "../../spec-spec") rtemsspec/tests/test_rtemsdocbuilder.py: os.path.join(os.path.dirname(__file__), "../../spec-spec"), rtemsspec/tests/test_filecopymanager.py: os.path.join(os.path.dirname(__file__), "../../spec-spec"), rtemsspec/tests/test_prequalifiedonly.py: os.path.join(os.path.dirname(__file__), "../../spec-spec"), rtemsspec/tests/test_prequalifiedonly.py: os.path.join(os.path.dirname(__file__), "../../spec-spec"), rtemsspec/tests/test_rsbbuilder.py: os.path.join(os.path.dirname(__file__), "../../spec-spec"), rtemsspec/tests/test_packagebuild.py: os.path.join(os.path.dirname(__file__), "../../spec-spec") ``` The directory `rtemsspec/tests/config-packagebuild/spec` which contains a duplicated configuration specification must go away. The copyright information and license should be derived from the items which contribute to the document and the header information of document source files. There are standard Python library methods to do things like this: ``` rtemsspec/tests/test_sddmanager.py: ret_info = subprocess.run(["ls", produced_doxy + "/sdd.pdf"], rtemsspec/tests/test_rsbbuilder.py: run_command(["cp", "-R", "working_dir/test", "installation"], cwd=tmpdir) rtemsspec/sddmanager.py: run_command(["mv", "refman.pdf", "../sdd.pdf"], latex_directory) rtemsspec/sddmanager.py: run_command(["rm", "-rf", "latex"], output_directory) ``` Common code should be moved to utility functions: ``` rtemsspec/tests/test_svsmanager.py: ret_info = subprocess.run(["diff", "-r", test_sources, produced_sources], rtemsspec/tests/test_sddmanager.py: ["diff", "-r", test_doxy, produced_doxy + "/html"], rtemsspec/tests/test_sddmanager.py: ret_info = subprocess.run(["diff", "-r", src_dir, working_dir_source], rtemsspec/tests/test_documentbuilder.py: ret_info = subprocess.run(["diff", "-r", test_sources, produced_sources], rtemsspec/tests/test_documentbuilder.py: ret_info = subprocess.run(["diff", "-r", test_sources, produced_sources], rtemsspec/tests/test_documentbuilder.py: ret_info = subprocess.run(["diff", "-r", test_sources, produced_sources], rtemsspec/tests/test_documentbuilder.py: ret_info = subprocess.run(["diff", "-r", test_sources, produced_sources], rtemsspec/tests/test_documentbuilder.py: ret_info = subprocess.run(["diff", "-r", test_sources, produced_sources], rtemsspec/tests/test_rtemsdocbuilder.py: ret_info = subprocess.run(["diff", "-r", test_depl, produced_depl], rtemsspec/tests/test_rtemsdocbuilder.py: ret_info = subprocess.run(["diff", "-r", test_src, produced_src], rtemsspec/tests/test_filecopymanager.py: ret_info = subprocess.run(["diff", "-r", test_depl, produced_depl], ``` Tests should not use the diff command. They should read the files and do a string comparison using assertions. This makes it a lot easier to figure out what is going on when a test fails.