I'm somewhat going round and round, first I wrote a shell script with plain txt files. Then I felt that shell script are not flexible enough so I wrote a python script for the work. Then I felt need more extensiblity than plain txt files go now I'm using YAML. I have made few small improvements in conda-skeleton. I want to track how each change improve my performance and hence I need a report card.
The should be something like this:
| Package | Recipe | Recipe log | Build | Build log |
|---|---|---|---|---|
| setuptools | True | setuptools_recipe.log | False | setuptool_build.log |
Score:
Recipe: 1/1
Build: 0/1Using Anaconda: True
Build Time: 12:30 20th May 2015 (IST)23:49 20th May
Yeah, building the reports was a great idea. Now the problems feels much more tractable. Each of the True or False in the recipe and build column is linked to the log. I was checking out the logs and most of the packages were failing due to or two missing dependency in the meta.yaml, I need to fix that. I'll try to parse the log file and build them, also I need to add option to only build the failing recipes again. Then if I'm left with other error I think I should first write a code to classify the error and list how many packages fall under that category.
Btw in the morning I somewhat struggled with subprocess, there is call and Popen. From my current understanding call executes the command and returns the status while Popen runs the process in background, hence Popen gives methods like poll, kill, pid etc. Both has keywords arguments stdout and stderr which takes file as an input. I wanted to get the output and error as strings. Popen has a method named communicate which returns the output of the command and the error and takes the input to the command, but everytime I was getting none because the command was already before I could run communicate so I used call with something like this:
err = subprocess.call(shlex.split(cmd), stdout=log_file,stderr=subprocess.STDOUT)
shlex is a lexical analyzer, according to its doc string, I'm not very sure what does that mean, but shlex.split splits the command string to into arguments which call accepts.