Updated gen_pack.sh script and accompanying documentation

CMSIS/DoxyGen/Pack/src/pack_check.txt

@@ -1,12 +1,12 @@
 /*=======0=========1=========2=========3=========4=========5=========6=========7=========8=========9=========0=========1====*/
 /**
-\page packChk packChk
+\page packChk PackChk
 
-\b packChk is a utility for validation of a Software Pack. It operates on the unzipped content of the Software Pack and
+\b PackChk is a utility for validation of a Software Pack. It operates on the unzipped content of the Software Pack and
 is located in the directory <b>.\\CMSIS\\Utilities</b> of the <b>ARM::CMSIS</b> Pack.  
-\b packChk is available for Windows and Linux operating systems.
+\b PackChk is available for Windows and Linux operating systems.
 
-\b packChk performs the following operations:
+\b PackChk performs the following operations:
  - Reads the content of the specified *.pdsc file. The path to this *.pdsc file is considered as root directory of the Software Pack.
  - Verifies the existence of all files in the Software Pack that are referenced in the *.pdsc file.
  - Checks for presence and correctness of mandatory elements such as \<vendor>, \<version>, etc.
@@ -18,13 +18,16 @@ is located in the directory <b>.\\CMSIS\\Utilities</b> of the <b>ARM::CMSIS</b>
      0 : no errors detected \n
      1 : errors during validation detected
 
+\note
+\b PackChk does not validate the XML conformance of your PDSC file. You need to run an additional XML validation tool such
+as \ref cp_Editors_xmllint "xmllint".
 
 Operation
 ---------
 
-\b packChk is invoked from the command line:\n
+\b PackChk is invoked from the command line:\n
 \code
- packChk <pdsc_file> <options>
+ PackChk <pdsc_file> <options>
 \endcode
 
 \n
@@ -48,7 +51,7 @@ Operation
     <td>-s</td>
     <td nowrap>Verify SVD files</td>	
     <td>Each SVD file referenced in the *.pdsc file is <a href="../../SVD/html/svd_validate_file_pg.html"><b>validated</b></a> with \b SVDConv.
-	With this option set, \b SVDConv.exe is invoked from the same directory as \b packChk.
+	With this option set, \b SVDConv.exe is invoked from the same directory as \b PackChk.
 	</td>
   </tr>
   <tr>
@@ -79,30 +82,30 @@ Operation
 
 <b>Examples</b>
 
-Run \b packChk on the package description file called <b>MyVendor.MyPack.pdsc</b>. \b packChk verifies the file against the Software Pack that is located in the same directory.
+Run \b PackChk on the package description file called <b>MyVendor.MyPack.pdsc</b>. \b PackChk verifies the file against the Software Pack that is located in the same directory.
 \verbatim
-packChk MyVendor.MyPack.pdsc
+PackChk MyVendor.MyPack.pdsc
 \endverbatim
 
-Run \b packChk on the package description file called <b>MyVendor.MyPack.pdsc</b> in the current directory and try to resolve conditions using the ARM.CMSIS.pdsc file based in another directory.
+Run \b PackChk on the package description file called <b>MyVendor.MyPack.pdsc</b> in the current directory and try to resolve conditions using the ARM.CMSIS.pdsc file based in another directory.
 \verbatim
-packChk MyVendor.MyPack.pdsc -i ..\..\PACK\ARM\CMSIS\ARM.CMSIS.pdsc
+PackChk MyVendor.MyPack.pdsc -i ..\..\PACK\ARM\CMSIS\ARM.CMSIS.pdsc
 \endverbatim
 
-Run \b packChk on the  package description file called <b>MyVendor.MVCM3.pdsc</b>, verify SVD files, verify the URL to the Pack Server, and generate a ASCII text file with the standardized name of the Software Pack.
+Run \b PackChk on the  package description file called <b>MyVendor.MVCM3.pdsc</b>, verify SVD files, verify the URL to the Pack Server, and generate a ASCII text file with the standardized name of the Software Pack.
 \verbatim
-packChk "MyVendor.MVCM3.pdsc" -s -u "http://www.myvendor.com/pack" -n packname.txt
+PackChk "MyVendor.MVCM3.pdsc" -s -u "http://www.myvendor.com/pack" -n packname.txt
 \endverbatim
 
-Run \b packChk on the  package description file called <b>MyVendor.MVCM3.pdsc</b>. Suppress validation messages M304 and M331.
+Run \b PackChk on the  package description file called <b>MyVendor.MVCM3.pdsc</b>. Suppress validation messages M304 and M331.
 \verbatim
-packChk MyVendor.MVCM3.pdsc -x M304 M331           // messages as a list
-packChk MyVendor.MVCM3.pdsc -x M304 -x M331        // option repeated
+PackChk MyVendor.MVCM3.pdsc -x M304 M331           // messages as a list
+PackChk MyVendor.MVCM3.pdsc -x M304 -x M331        // option repeated
 \endverbatim
 
 \section packChkMessages Error and Warning Messages
 
-The following table explains the categories for the output messages issued by packChk utility. Sections below list the errors and warnings and contain recommendations on how to resolve them.
+The following table explains the categories for the output messages issued by PackChk utility. Sections below list the errors and warnings and contain recommendations on how to resolve them.
 
 \b Categories
 \n
@@ -160,7 +163,7 @@ The errors in this category are issued because of an internal error in the PackC
 \anchor packchk_invoc_msgs 
 Invocation Errors
 -----------------
-The errors in this category are issued because of an incorrect command-line input that prevents the packChk execution.
+The errors in this category are issued because of an incorrect command-line input that prevents the PackChk execution.
 <table class="cmtable" summary="packChk Invocation Msgs">
   <tr>
     <th>Message Number</th>
@@ -175,7 +178,7 @@ The errors in this category are issued because of an incorrect command-line inpu
   <td>The list of arguments is too long. Verify if you have used one argument twice.</td>
 </tr>
 <tr><td>M202</td> <td>ERROR</td> <td>No PDSC input file specified</td>
-  <td>Correct the command line. \b packChk expects a \ref packFormat "*.pdsc" file name as input.</td>
+  <td>Correct the command line. \b PackChk expects a \ref packFormat "*.pdsc" file name as input.</td>
 </tr>
 <tr><td>M203</td> <td>ERROR</td> <td>Error reading PDSC file  \em 'PATH/FILENAME'!</td>
   <td>Verify the PDSC file for consistency.</td>
@@ -206,14 +209,14 @@ The errors in this category are issued because of an incorrect command-line inpu
 </tr>
 <tr><td>M211</td> <td>ERROR</td> <td>Cannot find SVDConv.exe: \em 'CMDLINE'</td>
   <td><a href="../../SVD/html/svd_SVDConv_pg.html">SVDConv utility</a> was not found in specified \em 'CMDLINE'.
-   <br>Copy the \b SVDConv.exe into the directory of \b packChk. Default is \token{CMSIS\\Utilities}.</td>
+   <br>Copy the \b SVDConv.exe into the directory of \b PackChk. Default is \token{CMSIS\\Utilities}.</td>
 </tr>
 </table>
 
 \anchor packchk_valid_msgs
 Validation Messages
 --------------------
-The messages in this category are issued by packChk during package validation. This can be for example incorrect use of the pack schema in the PDSC file, missing files, broken dependancies and others. \em ERROR messages must be resolved to ensure that the package is compliant to the CMSIS-format. \em WARNING messages should be checked and are strongly recommended to be resolved. 
+The messages in this category are issued by PackChk during package validation. This can be for example incorrect use of the pack schema in the PDSC file, missing files, broken dependancies and others. \em ERROR messages must be resolved to ensure that the package is compliant to the CMSIS-format. \em WARNING messages should be checked and are strongly recommended to be resolved. 
 
 <table class="cmtable" summary="packChk Validation Msgs">
   <tr>

CMSIS/DoxyGen/Pack/src/pack_utilities.txt

@@ -20,18 +20,52 @@ overview over programs that are either part of the CMSIS Pack or are available f
 
 Some text editors are capable of verifying XML code against a schema file. This is helpful to identify syntactical or
 structural problems in a PDSC or SVD file early in the development process. Useful editors are:
-- <a href="http://notepad-plus-plus.org/" target="_blank">Notepad++</a>
-- <a href="http://www.visualstudio.com/" target="_blank">Visual Studio (Express)</a>
+- <a href="https://notepad-plus-plus.org/" target="_blank">Notepad++</a>
+- <a href="https://code.visualstudio.com/" target="_blank">Visual Studio Code</a>
 
 Usually, the XML file that is to be validated and its corresponding schema file need to reside in the same directory. The
 CMSIS-Pack schema file \b Pack.xsd and the CMSIS-SVD schema file \b CMSIS-SVD.xsd are available in the directory
 <b>.\\CMSIS\\Utilities</b> of the <b>ARM::CMSIS</b> Pack.
 
-\anchor xmllint 
-<b>xmllint Schema Validation</b>
+\section cp_Editors_xmllint Schema Validation with xmllint
 
-The Linux utility <a href="http://xmlsoft.org/xmllint.html" target="_blank">xmllint</a> is a command line tool for XML validation against a schema file.
-It may be part of the \ref bash_script to verify the PDSC file during pack generation.
+The utility <a href="http://xmlsoft.org/xmllint.html" target="_blank">xmllint</a> is a command line tool for XML validation
+against a schema file. It may be part of the \ref bash_script to verify the PDSC file during pack generation.
+
+<b>Installing on Linux</b>
+
+Use your package manager to install the \c libxml2-utils (which contain xmllint). On a Ubuntu machine, use:
+\code
+$ sudo apt-get install libxml2-utils
+\endcode
+
+<b>Installing on Windows</b>
+
+Pre-built binaries of the libxml2-utils (which contain xmllint) are available through
+<a href="https://www.zlatkovic.com/projects/libxml/index.html" target="_blank">Libxml</a>. On the
+<a href="https://www.zlatkovic.com/pub/libxml/" target="_blank">download page</a>, retrieve the latest versions of \c iconv,
+\c libxml2, and \c zlib. Unzip the compressed files and copy the content of the \c /bin subdirectories onto your computer,
+for example to \c C:\\xmllint (this directory is set in \ref bash_script "gen_pack.sh"). In the end, the content of this
+directory should look like the following:
+\code
+C:\xmllint>dir
+ Volume in drive C is Harddisk
+ Volume Serial Number is ABBA-AFFE
+ 
+ Directory of C:\xmllint
+ 
+29/02/2020  13:14    <DIR>          .
+29/02/2020  13:14    <DIR>          ..
+03/05/2006  21:57           888,832 iconv.dll
+03/05/2006  21:57             7,680 iconv.exe
+19/06/2011  21:52           987,136 libxml2.dll
+02/10/2010  13:21            57,344 minigzip.exe
+19/06/2011  21:53            10,240 xmlcatalog.exe
+19/06/2011  21:53            38,912 xmllint.exe
+02/10/2010  13:21            77,824 zlib1.dll
+               7 File(s)      2,067,968 bytes
+               2 Dir(s)  212,703,227,904 bytes free
+\endcode
 */
 
 /*=======0=========1=========2=========3=========4=========5=========6=========7=========8=========9=========0=========1====*/
@@ -42,25 +76,46 @@ Any ZIP software utility supporting the compression method
 <a href="http://en.wikipedia.org/wiki/DEFLATE" target="_blank">DEFLATE</a> can be used to create a Pack file. Simply, create
 the ZIP file and rename it to *.pack.
 
+\section cp_ZIPTool_7z 7-Zip
+
 The compression tool <a href="http://www.7-zip.org/" target="_blank">7-Zip</a> supports command line calls and can be
-used in batch scripts like \ref cp_GeneratePack "gen_pack.bat" for automated Pack file creation (and verification
-with \ref packChk). Installers for 7-Zip are available for 32-bit and 64-bit Windows operating systems.
+used in generation scripts like \ref bash_script "gen_pack.sh" for automated Pack file creation (and verification
+with \ref packChk).
+
+<b>Installing on Linux</b>
+
+Use your package manager to install <a href="http://www.7-zip.org/" target="_blank">7-Zip</a>. On a Ubuntu machine, use:
+\code
+$ sudo apt-get install p7zip-full
+\endcode
+
+<b>Installing on Windows</b>
+
+In the download section of <a href="http://www.7-zip.org/" target="_blank">7-Zip</a>, download the appropriate installer
+for your Windows system. Use defaults for your installation.
+
+\ref bash_script "gen_pack.sh" assumes that your installation path is \c "C:\Program Files\7-Zip". If you choose to use
+another location, please adapt the path in the Bash script.
 */
 
 /*=======0=========1=========2=========3=========4=========5=========6=========7=========8=========9=========0=========1====*/
 /**
 \page bash_script Bash Script gen_pack.sh
 
-The following <a href="https://en.wikipedia.org/wiki/Bash_(Unix_shell)" target="_blank">Bash</a> shell script template \ref gen_pack_sh "gen_pack.sh" allows you to generate
-a Pack file on the Linux or the Windows operating system.  Bash is available for Windows, for example, via <a href="https://gitforwindows.org/" target="_blank">git for Windows</a>.
+The following <a href="https://en.wikipedia.org/wiki/Bash_(Unix_shell)" target="_blank">Bash</a> shell script template
+\ref gen_pack_sh "gen_pack.sh" allows you to generate a pack file under Linux or Windows. On Windows, Bash is available via
+<a href="https://gitforwindows.org/" target="_blank">git for Windows</a> for example.
 
 
 For using \ref gen_pack_sh "gen_pack.sh" 
- - Install \ref cp_ZIPTool "7-Zip" (and for Linux \ref xmllint "xmllint" for schema validation).
- - Adapt the file \ref gen_pack_sh "gen_pack.sh" that is available in the directory <b>/CMSIS/Pack/Bash</b> to the requirements of your software pack.
+ - Install \ref cp_ZIPTool "7-Zip".
+ - Install \ref cp_Editors_xmllint "xmllint".
+ - Adapt the file \ref gen_pack_sh "gen_pack.sh" that is available in the directory <b>/CMSIS/Pack/Bash</b> to the
+   requirements of your software pack and .
 
-The script is generic enough to cope with a wide range of requirements.  It uses the information from the \ref packFormat "*.PDSC file" to generate 
-the output filename according the CMSIS-Pack conventions and validates the pack consistency with \ref packChk.
+The script is generic enough to cope with a wide range of requirements. It uses the information from the
+\ref packFormat "*.PDSC file" to generate the output filename according the CMSIS-Pack conventions and validates the pack
+consistency with \ref packChk.
 
 Below is a sample output (reduced).
 \code
@@ -105,10 +160,10 @@ Completed CMSIS-Pack Generation ...
 \anchor gen_pack_sh
 <b>gen_pack.sh Bash script template file</b>
 
-The <b>gen_pack.sh</b> script template allows to configure:
- - Path environment variables for related utilities
- - Directory names for temporary build and output files
- - Directory names and files in the root directory that should be included in the pack
+The <b>gen_pack.sh</b> script template allows you to configure:
+ - Path environment variables for related utilities (\c CMSIS_PACK_PATH, \c PATH_TO_ADD)
+ - Directory names for temporary build and output files (\c PACK_WAREHOUSE, \c PACK_BUILD)
+ - Directory names and files in the root directory that should be included in the pack (\c PACK_DIRS, \c PACK_BASE_FILES)
  
 
 \verbinclude gen_pack.sh

CMSIS/Pack/Bash/gen_pack.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
-# Version: 1.0 
-# Date: 2019-08-16
+# Version: 1.1 
+# Date: 2020-04-29
 # This bash script generates a CMSIS Software Pack:
 #
 # Pre-requisites:
@@ -9,18 +9,20 @@
 #   e.g. Ubuntu: sudo apt-get install p7zip-full p7zip-rar) 
 # - PackChk in path with execute permission
 #   (see CMSIS-Pack: CMSIS/Utilities/<os>/PackChk)
-# - xmllint in path (XML schema validation; available only for Linux)
+# - xmllint in path (XML schema validation)
+#   e.g. Ubuntu: sudo apt-get install libxml2-utils
+#   Windows: download from https://www.zlatkovic.com/pub/libxml/
 
 ############### EDIT BELOW ###############
 # Extend Path environment variable locally
 #
 if [ `uname -s` = "Linux" ]
   then
-  CMSIS_PACK_PATH="/home/$USER/.arm/Packs/ARM/CMSIS/5.6.0/"
-  PATH_TO_ADD="$CMSIS_PACK_PATH/CMSIS/Utilities/Linux-gcc-4.8.3/"
+  CMSIS_PACK_PATH="/home/$USER/.arm/Packs/ARM/CMSIS/5.7.0/"
+  PATH_TO_ADD="$CMSIS_PACK_PATH/CMSIS/Utilities/Linux64/"
 else
-  CMSIS_PACK_PATH="/C/Keil_v5/ARM/PACK/ARM/CMSIS/5.6.0"
-  PATH_TO_ADD="/C/Program Files/7-Zip/:$CMSIS_PACK_PATH/CMSIS/Utilities/Win32/"
+  CMSIS_PACK_PATH="$LOCALAPPDATA/Arm/Packs/ARM/CMSIS/5.7.0"
+  PATH_TO_ADD="/C/Program Files/7-Zip/:$CMSIS_PACK_PATH/CMSIS/Utilities/Win32/:/C/xmllint/"
 fi
 [[ ":$PATH:" != *":$PATH_TO_ADD}:"* ]] && PATH="${PATH}:${PATH_TO_ADD}"
 echo $PATH_TO_ADD appended to PATH
@@ -80,6 +82,19 @@ if [ $errorlevel != 0 ]
 fi
 echo " "
 
+# XML syntax checking utility check
+XMLLINT=xmllint
+type -a $XMLLINT
+errorlevel=$?
+if [ $errorlevel != 0 ]
+  then
+  echo "Error: No xmllint found"
+  echo "Action: Add xmllint to your path"
+  echo " "
+  exit
+fi
+echo " "
+
 # Locate Package Description file
 # check whether there is more than one pdsc file
 NUM_PDSCS=`ls -1 *.pdsc | wc -l`
@@ -139,18 +154,13 @@ done
 # Run Schema Check (for Linux only):
 # sudo apt-get install libxml2-utils
 
-if [ `uname -s` = "Linux" ]
-  then
-  echo Running schema check for $PACK_VENDOR.$PACK_NAME.pdsc
-  xmllint --noout --schema ${CMSIS_PACK_PATH}/CMSIS/Utilities/PACK.xsd $PACK_BUILD/$PACK_VENDOR.$PACK_NAME.pdsc
-  errorlevel=$?
-  if [ $errorlevel -ne 0 ]; then
-    echo "build aborted: Schema check of $PACK_VENDOR.$PACK_NAME.pdsc against PACK.xsd failed"
-    echo " "
-    exit
-  fi
-else
-  echo "Use MDK PackInstaller to run schema validation for $PACK_VENDOR.$PACK_NAME.pdsc"
+echo Running schema check for $PACK_VENDOR.$PACK_NAME.pdsc
+$XMLLINT --noout --schema ${CMSIS_PACK_PATH}/CMSIS/Utilities/PACK.xsd $PACK_BUILD/$PACK_VENDOR.$PACK_NAME.pdsc
+errorlevel=$?
+if [ $errorlevel -ne 0 ]; then
+  echo "build aborted: Schema check of $PACK_VENDOR.$PACK_NAME.pdsc against PACK.xsd failed"
+  echo " "
+  exit
 fi
 
 # Run Pack Check and generate PackName file with version