Copyright (C) 2009-2016 Institut de Robòtica i Informàtica Industrial, CSIC-UPC.
Author David Martínez (dmartinez@iri.upc.edu)
All rights reserved.

This file is part of REX-D library
REX-D library is free software: you can redistribute it and/or modify
it under the terms of the GNU Lesser General Public License as published by
the Free Software Foundation, either version 3 of the License, or
at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>



===========================
===Questions/Suggestions===
===========================
If you have any question or suggestion, don't hesitate to contact me:
email: dmartinez@iri.upc.edu



===========================
===    Documentation    ===
===========================
Online documentation can be found at: 
www.iri.upc.edu/people/dmartinez/rexd_docs/index.html



======================
===  Introduction  ===
======================
REX-D is a relational reinforcement and active learning algorithm.

When using this code, if using REX-D please cite:
Relational reinforcement learning with guided demonstrations, David Martínez, 
Guillem Alenyà, Carme Torras. Artificial Intelligence, 2016

If using the domain model learner, please cite:
Learning Relational Dynamics of Stochastic Domains for Planning, David Martínez, 
Guillem Alenyà, Carme Torras, Tony Ribeiro and Katsumi Inoue. In proc. of the
International Conference on Automated Planning and Scheduling, 2016, pp. 235-243



======================
=== Installation   ===
======================
Installation has been tested in Ubuntu 14.04. Should be compatible with other 
linux distributions. Windows and OSX are not supported.

REX-D requires CMake and Boost to run, which should be available in your linux 
distribution. In Ubuntu run 
$ sudo apt-get install build-essential cmake cmake-curses-gui subversion \
libboost-dev libboost-system-dev libboost-filesystem-dev libboost-regex-dev \
libboost-program-options-dev libboost-thread-dev

Go to the REX-D root folder. To compile create a build directory and run cmake:
$ mkdir build
$ cd build
$ cmake ..

By default, just the REX-D library is enabled. To enable the example app do:
$ ccmake .
Go to COMPILE_APPS and set it to ON (press enter)
Press 'c' and 'g'

Now build everything:
$ make

--------------------
Planner installation
--------------------
Although the prada planner is installed by default, it is highly recommendable 
to install either Gourmand or PROST planners which require just a few steps.
(Note that all the example domains use the Gourmand or PROST by default.)

------------
* Gourmand *
Requires flex and bison. In Ubuntu they can be installed with:
$ sudo apt-get install flex bison

Go to the Gourmand planner directory from the REX-D root folder:
$ cd external/G-pack/
$ make
Copy the planner to the bin folder
$ cp planner ../../bin/gpack

---------
* PROST *
Requires BDD library BuDDy. In Ubuntu it can be installed with:
$ sudo apt-get install libbdd-dev

Go to the PROST planner directory from the REX-D root folder:
$ cd external/PROST/src/search
$ make
Copy the planner to the bin folder
$ mkdir ../../../../bin/prost/
$ cp prost ../../../../bin/prost/

Go to the PROST RDDL parser directory from the REX-D root folder:
$ cd external/PROST/src/rddl_parser
$ make
Copy the parser to the bin folder
$ cp rddl-parser ../../../../bin/prost/


--------------------
Batch learner installation
--------------------
Although the Pasula et al's learner is installed by default, a learner that 
can tackle domain with dynamic effects can be used. However, it requires 
the LFIT learner that has to be installed as follows.
(Note that all the example domains use the LFIT-based learner.)

Go to the LFIT learner directory from the REX-D root folder:
$ cd external/LFIT
$ qmake LFIT.pro
$ make
Copy the learner to the bin folder
$ cp bin/LFIT ../../bin/lfit



==============================
===  REX-D / V-MIN Usage   ===
==============================
Use RL with a teacher to solve a problem (and learn it).

Go to the scenarios folder.
$ cd scenarios

Run the REX-D application on a problem
Problems are organized as follows: 
 - subfolders in the scenarios/domains folder are the different domains
 - subfolders in each domain are the problems.
 - Example: domains/triangle/p1rmin/ is domain triangle and problem p1vmin
$ ../bin/execution_scripts/make_experiment.py domains/triangle/p1vmin/

In another terminal run the simulator
$ cd scenarios
$ ../bin/simulator.sh



===================================
=== Domain Model Learner Usage  ===
===================================
Learn a domain model from a set of input experiences.

Go to the scenarios folder.
$ cd scenarios

Run the learner on a problem
Problems are organized as follows: 
 - subfolders in the scenarios/domains folder are the different domains
 - subfolders in each domain are the problems.
 - Example: domains/triangle/p1rmin/ is domain triangle and problem p1vmin
 
Copy problem files
$ ../bin/execution_scripts/start_experiment.py domains/triangle/p1lfit
$ cd current
Generate a set of 15 random transitions for each action to be used as input of the learner
$ bin/learner_utils transitions 15
Run the learner
$ bin/learner_utils learn
Evaluate rules with 2000 random transitions per action
$ bin/learner_utils evaluate_rules learned_rules.ppddl 2000

Note that the provided version of LFIT can be very slow with the Elevators domain. Our aim is to have a faster one in the short future.



======================
===  New domains   ===
======================
Domain should be defined in PPDDL. See examples in scenarios/domains folder.



======================
===  Experiments   ===
======================
To run repeated experiments move to the scenarios folder and use the scripts:
Run the REX-D application (infinite loop of experiments, ctrl+c to stop)
$ cd scenarios
$ ../bin/rexd.sh domains/domain/problem
and the simulator (in a new terminal)
$ cd scenarios
$ ../bin/simulator.sh



======================
===  MATLAB PLOTS  ===
======================
Copy results/example folder
Inside create a folder for each configuration, and copy there all the 
corresponding experiment_<scenario>_<problem>_<num>.

[ Note that experiment_<scenario>_<problem>_<num> folders are created in the 
scenarios/results/ folder when running bin/make_experiment_simulator.py or 
bin/rexd.sh scripts. ]

Inside the copy of the results/example folder run:
../get_exp_info.py > graphs/generated_times.h
Open graphs/create_graphs.m in matlab
Search and replace each exp* for the name of folder created for each 
configuration. Using the same folders as in the results/example, it would be 
replacing "exp0" for "config1" and "exp1" for "config2".



======================
===      ROS       ===
======================
To be provided in the future.
Contact me (dmartinez@iri.upc.edu) if you need it.



======================
===     TESTS      ===
======================
Requires gtest installed and compiled
http://askubuntu.com/questions/145887/why-no-library-files-installed-for-google-test-on-12-04

Enable COMPILE_TESTS in cmake (you can use "ccmake ." as in the installation)

Go to tests folder and run:
$ ../bin/gtests


======================
===  DOXYGEN DOCS  ===
======================
Install doxygen
$ sudo apt-get install doxygen

Enable GENERATE_DOCS in cmake (you can use "ccmake ." as in the installation)

In build directory execute:
$ make doc


======================
===Video generation===
======================
Video example: TODO I will update a video example in the future
WARNING: this is just a optional feature and is not trivial to make it work
Requires: NVidia graphics card

# Gource
# Note: requires nvidia graphic card
Download gource-0.40
Patch with external/gource.0.40.patch
Compile and install

# Avxsynth, ffmpeg & stuff
# Note: add --require-libass and --require-libx264 to ffmpeg 
# (and install required -dev packages)
# Also recommended mplayer2
http://forum.doom9.org/showthread.php?p=1643184#post1643184

# Run script 
<REXD>/bin/video_gource_offline.sh path_to_experiment
# Change speed_modifier in avs file
# Run script 
<REXD>/bin/video_file_processing.sh