GI Magpie

FileNotFoundError: [Errno 2] No such file or directory: '.../_magpie_work/project_time/project

==== WARMUP ====
==== END ====
Reason: None
Traceback (most recent call last):
  File ".../magpie/base/program.py", line 193, in sync_folder
    contents_target = os.listdir(target)
FileNotFoundError: [Errno 2] No such file or directory: '.../_magpie_work/.../...'
This error can arise as Magpie tried to create its own work directory _magpie_work as a copy of your direcory (given by path in the scenario file) and a file is open in your directory.

Early versions of Magpie

The following relate to problems in Magpie in 2022. Some may have been resolved by more recent versions of Magpie.

Initial software failed to run! CMD: ['pytest']

Add pytest to $PATH

Have you got pytest installed? Is it in your path? Try running pytest on the command line: 
pytest --version
pytest: Command not found.

If the operating system cannot find pytest, perhaps add it to your system environment PATH variable. When in unix using the "T"shell command line interpreter, tcsh, try something like: 

setenv PATH /opt/Python/Python-3.7/bin:"$PATH"
NB: make sure not to upset the version of Python you are using (see text below).

Pytest Slow

Try Again

Particularly on networked computers it may take an excessively long (many seconds) for pytest to start the first time.

Sometimes the easiest solution is simply to try your Magpie command again.

Preload Pytest

Otherwise perhaps force pytest to run once succesfully outside Magpie before trying to start Magpie. Even pytest --version might be sufficient to get the operating system warmed up so that pytest starts immediately when Magpie tries to use it.

minify_patch Initial software failed to run! CMD: ['python', 'run_triangle.py']

WARM    PARSE_ERROR       0                       
Initial software failed to run!
CMD:
['python', 'run_triangle.py']
Did you use the wrong --mode with python3 -m bin.minify_patch?

Wrong --mode

Are you using bin.minify_patch with the wrong --mode=? E.g. repair when it should be --mode=runtime?

KeyError: 'software'

python3 -m bin.magpie_runtime --config examples/scenario/triangle-c_runtime.txt
fails with

self.base_init(config['software']['path']) ... KeyError: 'software'

--config file missing

Correct --config file name or path to it.

KeyError: 'test_cmd'

python3 -m bin.magpie_runtime --config examples/triangle-c_runtime.txt
fails with
self.test_cmd = config['software']['test_cmd'] ... KeyError: 'test_cmd'

Possible error in --config file

Check contents of --config file, particularly test_cmd = and path =

python3 -m bin.magpie_runtime --config examples/scenario/triangle-py_runtime.txt stops too soon

Remove time limit

Edit max_time = 60 in triangle-py_runtime.txt Perhaps set larger upper limit. Perhaps remove or comment out entirely (by inserting # at start of line).
    Without max_time magpie will terminate on [search] parameter max_steps (line above max_time = 60).

python3 -m bin.magpie_runtime --config examples/scenario/triangle-py_runtime.txt gives poor results

Variability

GI is a stochastic search, so fantastic solutions can not be guaranteed. Since by default fitness depends on run time, a possible cause is variability between elapse time on testing mutants. This seems to especially effect networked computers. Perhaps check variation in the ==== WARMUP ==== reported at the start of magpie run. The same program is run four times, if the four elapsed times (i.e. fitness) are very different, especially is the last two or three are different, there may be too much variation in run time on your computer to get a good fitness signal for each mutant to drive evolution to a satisfactory solution.

Possible approaches:

Results hard to interpret

At the end of the run magpie ==== REPORT ==== says
  1. Termination: why it stopped, e.g. because it reached a search limit (e.g. max_steps in the --config file).
  2. Log file: log file contains many details of the magpie run.
  3. Best fitness:
  4. Best patch: Individual genes in the best overall patch are separated by vertical bars |. The best patch is also saved in _magpie_logs/triangle-py_slow_??????????.patch.
  5. Diff: (also reported in _magpie_logs/triangle-py_slow_??????????.diff).
With experience magpie patches can be interpreted. Similarly, if you are already familiar with code patches, the Diff corresponding to the best fitness found by the end of the run, can also be interpreted.
    An alternative is to use the operating system to automatically apply the Diff found by magpie to the original program source code. This will give exactly the mutated code that magpie has just reported as being the best it found.

Note: magpie reports line n start at zero (not one).

Apply patch

Useful patch options 
cp -p examples/code/triangle-py_slow/triangle.py .
patch --backup --input=_magpie_logs/triangle-py_slow_1669990485.diff triangle.py
triangle.py will now contain the patched python code. triangle.py.orig is the original file.
    Remember magpie line numbers start at 0.

Example

triangle.py.orig 
 0:import time
 1:from enum import Enum
 2:
 3:
 4:class TriangleType(Enum):
 5:    INVALID, EQUILATERAL, ISOCELES, SCALENE = 0, 1, 2, 3
 6:
 7:
 8:def delay():
 9:    time.sleep(0.001)
10:
11:
12:def classify_triangle(a, b, c):
13:
14:    delay()
15:
16:    # Sort the sides so that a <= b <= c
17:    if a > b:
18:        tmp = a
19:        a = b
20:        b = tmp
21:
22:    if a > c:
23:        tmp = a
24:        a = c
25:        c = tmp
26:
27:    if b > c:
28:        tmp = b
29:        b = c
30:        c = tmp
31:
32:    if a + b <= c:
33:        return TriangleType.INVALID
34:    elif a == b and b == c:
35:        return TriangleType.EQUILATERAL
36:    elif a == b or b == c:
37:        return TriangleType.ISOCELES
38:    else:
39:        return TriangleType.SCALENE

Best patch: 
LineInsertion(('triangle.py', '_inter_line', 31), ('triangle.py', 'line', 7)) | 
LineInsertion(('triangle.py', '_inter_line', 33), ('triangle.py', 'line', 21)) | 
LineReplacement(('triangle.py', 'line', 9), ('triangle.py', 'line', 37)) | 
LineInsertion(('triangle.py', '_inter_line', 4), ('triangle.py', 'line', 7))
  1. LineInsertion(('triangle.py', '_inter_line', 31), ('triangle.py', 'line', 7))
    Line, 7 (a blank line) is inserted at line 31.
  2. LineInsertion(('triangle.py', '_inter_line', 33),('triangle.py', 'line', 21))
    Line, 21 (another blank line) is inserted at line 33.
  3. LineReplacement(('triangle.py', 'line', 9), ('triangle.py', 'line', 37))
    Line, 37 (return TriangleType.ISOCELES) replaces line 9.
    Notice line 9 is in function delay() and LineReplacement effectively destroys delay()'s functionality by overwriting it's call to sleep with an immediate return statement. Thus when delay() is called, it no longer slows down the program.
  4. LineInsertion(('triangle.py', '_inter_line', 4), ('triangle.py', 'line', 7))
    Line, 7 (which is still a blank line) is inserted at line 4.
Patched triangle.py 
 0:import time
 1:from enum import Enum
 2:
 3:
 4:
 5:class TriangleType(Enum):
 6:    INVALID, EQUILATERAL, ISOCELES, SCALENE = 0, 1, 2, 3
 7:
 8:
 9:def delay():
10:        return TriangleType.ISOCELES
11:
12:
13:def classify_triangle(a, b, c):
14:
15:    delay()
16:
17:    # Sort the sides so that a <= b <= c
18:    if a > b:
19:        tmp = a
20:        a = b
21:        b = tmp
22:
23:    if a > c:
24:        tmp = a
25:        a = c
26:        c = tmp
27:
28:    if b > c:
29:        tmp = b
30:        b = c
31:        c = tmp
32:
33:
34:    if a + b <= c:
35:
36:        return TriangleType.INVALID
37:    elif a == b and b == c:
38:        return TriangleType.EQUILATERAL
39:    elif a == b or b == c:
40:        return TriangleType.ISOCELES
41:    else:
42:        return TriangleType.SCALENE




diff triangle.py.orig triangle.py
4a5
> 
10c11
<     time.sleep(0.001)
---
>         return TriangleType.ISOCELES
32a34
> 
33a36
>

Results are wrong

At the end of its run, magpie displays the best patch it has found and data about it, including its fitness. In one case, the best patch was clearly broken and triangle.c should not have compiled. (NB remember magpie lines start at 0. Perhaps try applying the patch, see section above, and then maybe use standalone.c to double check.)

Test suite is insufficient

Sometimes magpie finds patches which are incorrect but pass all the tests.

Unix tcsh script standalone.bat may help confirm that patched triangle.c does indeed pass the test suite. (See text above on automatically generating the magpie patched code.)

Replace make

The RCS documentation (pages 12-13) says "MAKE has been extended with an auto-checkout feature for RCS.* When a certain file to be processed is not present, MAKE attempts a check-out operation. If successful, MAKE performs the required processing, and then deletes the checked out file to conserve space.

*This auto-checkout extension is available only in some versions of MAKE, e.g. GNU MAKE."

It may be this leads to an unfortunate interaction with magpie. In any case it seems simpler and safer to replace compile_cmd = make with the corresponding compilation commands. For triangle.c when using the GNU GCC compiler this is compile_cmd = gcc -shared -o triangle.so -fmax-errors=1 -fPIC triangle.c

Since magpie copies examples/code/triangle-c_slow and its sub directories, it seems also wise to move the corresponding RCS sub-directory elsewhere too.

Patch Minification

The previous example shows that magpie need not find the "best" possible patch and the patches it finds can contain unnecessary changes. However it is possible to use magpie's minify_patch to automatically slim patches. 
python3 -m bin.minify_patch --mode=runtime --config examples/scenario/triangle-py_runtime.txt --patch "LineInsertion(('triangle.py', '_inter_line', 31), ('triangle.py', 'line', 7)) | LineInsertion(('triangle.py', '_inter_line', 33), ('triangle.py', 'line', 21)) | LineReplacement(('triangle.py', 'line', 9), ('triangle.py', 'line', 37)) | LineInsertion(('triangle.py', '_inter_line', 4), ('triangle.py', 'line', 7))"
Note minify_patch is still using actual program run time to guide its search and so is still subject to noise (as described in variability above).

In one run, minify_patch (due to noise) reported the best patch it could find still contained two changes:
Best patch: LineReplacement(('triangle.py', 'line', 9), ('triangle.py', 'line', 37)) | LineInsertion(('triangle.py', '_inter_line', 33), ('triangle.py', 'line', 21))

Again we can use the unix patch command (see description of patch above) to automatically apply the new patch created by magpie: 

patch --backup --input=_magpie_logs/triangle-py_slow_1670000225.diff triangle.py                 
patching file triangle.py

diff triangle.py.orig triangle.py
10c10
<     time.sleep(0.001)
---
>         return TriangleType.ISOCELES
33a34
>
LineReplacement 9←37 is explained above. Notice due to noise, we are still have LineInsertion 33←21, even though all it does is add an empty line. (Again remember magpie line numbers start at zero.)

AttributeError: module 'time' has no attribute 'time_ns'

Which version of Python are you running? time_ns() needs at least python 3.8 Try python3 --version and/or which python

Add newer Python to $PATH

With tcsh, perhaps try something like: 
setenv PATH /opt/Python/Python-3.10.1/bin:"$PATH"
NB: make sure not to upset the version of pytest you are using (see text above).

NameError: name 'median_high' is not defined

Which version of Python are you running?

median_high() median_low and quantiles() were added to the statistics module and so you need at least python 3.8 See previous text.

python3: error while loading shared libraries: libpython3.10.so.1.0: cannot open shared object file: No such file or directory

It may be your $LD_LIBRARY_PATH unix environment variable does not point to a shared run time library file like libpython3.10.so Perhaps try echo $LD_LIBRARY_PATH

Add libpython3.10.so to $LD_LIBRARY_PATH

Find the location of a suitable version of the python shared runtime support library and then add its location to your LD_LIBRARY_PATH unix system environment variable. With tcsh, perhaps try something like: 
setenv LD_LIBRARY_PATH  /opt/Python/Python-3.10.1/lib:"$LD_LIBRARY_PATH"

NameError: name 'quantiles' is not defined

See NameError: text above.

NameError: name 'floor' is not defined. Did you mean: 'float'?

import floor() from Python math module

Perhaps add something like the following to your python code: 
from math import floor

Python coding problems

NameError: name 'sanity' is not defined

def sanity(x): in wrong place

If not part of class, move def sanity(x): to before where it is called.

Did not put class name or self. in front of function call

If function is part of same class, try something like self.sanity(x)

TypeError: Program.sanity() takes 1 positional argument but 2 were given

Forgot self as first argument for function def

Try 
    def sanity(self, a):

Unix patch problem

patch eg Hunk #3 FAILED at 142.

Example: 
patch --input=xyz-c_1672736071.diff xyz.c
patching file xyz.c
Hunk #2 succeeded at 127 with fuzz 1.
Hunk #3 FAILED at 142.
Hunk #4 FAILED at 150.
Hunk #5 FAILED at 160.
Hunk #6 FAILED at 169.
Hunk #8 succeeded at 309 with fuzz 1.
Hunk #9 succeeded at 344 with fuzz 2.
Hunk #12 succeeded at 1390 with fuzz 1.
Hunk #13 succeeded at 1417 with fuzz 2.
Hunk #17 FAILED at 1496.
5 out of 19 hunks FAILED -- saving rejects to file xyz.c.rej

Increase patch fuzz

Sometimes patch can be persuaded to work with the .diff files produced by Magpie by using patch --fuzz=3 (see unix man patch).

Apply Magpie Patches by Hand

Sometimes on long and complicated patches automatically generated by Magpie, increasing patch fuzz=n does not work. In such cases the source file can be edited by hand, e.g. using the Magpie .log file as a guide.
W.B.Langdon Back Started 2 December 2022. Last update $Date: 2023/10/01 15:06:49 $