为了尝试一下 cs294 课程的作业，配环境配了一天多。
Deep RL Assignment 1: Imitation Learning Fall 2017 作业要求上一句轻描淡写：
- MuJoCo: We will use MuJoCo for physics simulation in this assignment. Download version 1.31 from mujoco.org and obtain a license from the course instructors. Note that we use version 1.31 and not the latest version 1.50. The python interface to mujoco bundled with OpenAI gym is also compatible with this version of mujoco.
前面一直出 bug 的是 gym：
Traceback (most recent call last):
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\gym\envs\registration.py", line 143, in spec
return self.env_specs[id]
KeyError: 'Humanoid-v1'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "run_expert.py", line 71, in <module>
main()
File "run_expert.py", line 37, in main
env = gym.make(args.envname)
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\gym\envs\registration.py", line 167, in make
return registry.make(id)
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\gym\envs\registration.py", line 118, in make
spec = self.spec(id)
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\gym\envs\registration.py", line 151, in spec
raise error.DeprecatedEnv('Env {} not found (valid versions include {})'.format(id, matching_envs))
gym.error.DeprecatedEnv: Env Humanoid-v1 not found (valid versions include ['Humanoid-v2'])
感谢最后一句的提醒，估计是什么更新了吧，v1没有了，我换成了 v2，这一句终于能过掉了。由于还没有装 mujoco，错误变成了：
(cs294hw1) C:\ProjectsData\machineLearning\ACMCourse2\cs294\homework\hw1>python run_expert.py experts\Humanoid-
v1.pkl Humanoid-v2 --render --num_rollouts 20
loading and building expert policy
obs (1, 376) (1, 376)
loaded and built
……（此处略去几十行）
Traceback (most recent call last):
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\gym\envs\mujoco\mujoco_env.py", line 11, in <module>
import mujoco_py
ModuleNotFoundError: No module named 'mujoco_py'
安装 mujoco_py 和 mujoco
我开始安装 mujoco_py。据 博客 提示，1.3.1 mjpro对应的0.5.7py，我就去 pip3.5 install mujoco_py==0.5.7 ，一切顺利。
然后 import mujoco_py时，发生错误：
>>> import mujoco_py
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\mujoco_py\__init__.py", line 2, in <module>
init_config()
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\mujoco_py\config.py", line 33, in init_config
raise error.MujocoDependencyError('To use MuJoCo, you need to either populate ~/.mujoco/mjkey.txt and ~/.mujoco/mjpro131, or set the MUJOCO_PY_MJKEY_PATH and MUJOCO_PY_MJPRO_PATH environment variables appropriately. Follow the instructions on https://github.com/openai/mujoco-py for where to obtain these.')
mujoco_py.error.MujocoDependencyError: To use MuJoCo, you need to either populate ~/.mujoco/mjkey.txt and ~/.mujoco/mjpro131, or set the MUJOCO_PY_MJKEY_PATH and MUJOCO_PY_MJPRO_PATH environment variables appropriately. Follow the instructions on https://github.com/openai/mujoco-py for where to obtain these.
好吧，我确实还没有找 key。我到官网注册了MuJoCo Pro Trial License: 30 days，从邮箱下载了 LICENSE.txt 和 mjkey.txt。同时在官网 下载了mjpro131 win64，解压到了C:\Program Files\mujoco文件夹。我不仅把两个文件疯狂地复制到了C:\Program Files\mujoco、C:\Program Files\mujoco\mjpro131 以及 C:\Program Files\mujoco\mjpro131\bin，系统环境变量也加进去了，如下图，（这里其实还写错了，KEY的应该具体到文件）结果还是不行。
这都不行，issue里也搜索不到有帮助的信息，那，只能，去改掉报错的文件了（微笑）。我去看了文件C:\anaconda3\envs\cs294hw1\Lib\site-packages\mujoco_py\config.py，发现写道 _key_path = os.environ.get('MUJOCO_PY_MJKEY_PATH')
自己尝试这句话，果然是拿不到的。
（后来发现还是我蠢，我没有重启一下命令行，好像重开一个才会拿到新的环境变量= =这样的话应该是只要添加了环境变量就OK了）
C:\Users\shens
λ python
Python 3.6.3 (v3.6.3:2c5fed8, Oct 3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> _key_path = os.environ.get('MUJOCO_PY_MJKEY_PATH')
>>> print(_key_path)
None
然后下面给了 default 值竟然是按照 Linux 的写法写的，怪不得没有这个路径……为了杜绝后患，我把这一大段 if 不 if 的都改掉了，直接暴力赋值
_key_path = os.path.expanduser('C:\\Program Files\\mujoco\\mjpro131\\bin\\mjkey.txt')
mjpro_path = os.path.expanduser('C:\\Program Files\\mujoco\\mjpro131')
这下终于过了，不过报错变成了
(cs294hw1) C:\ProjectsData\machineLearning\ACMCourse2\cs294\homework\hw1>python
Python 3.6.2 |Continuum Analytics, Inc.| (default, Jul 20 2017, 12:30:02) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import mujoco_py
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\mujoco_py\__init__.py", line 4, in <module>
from .mjviewer import MjViewer
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\mujoco_py\mjviewer.py", line 7, in <module>
from . import mjcore, mjconstants, glfw
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\mujoco_py\mjcore.py", line 6, in <module>
from .mjlib import mjlib
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\mujoco_py\mjlib.py", line 21, in <module>
mjlib = cdll.LoadLibrary(os.path.abspath(libfile))
File "C:\anaconda3\envs\cs294hw1\lib\ctypes\__init__.py", line 426, in LoadLibrary
return self._dlltype(name)
File "C:\anaconda3\envs\cs294hw1\lib\ctypes\__init__.py", line 348, in __init__
self._handle = _dlopen(self._name, mode)
OSError: [WinError 193] %1 不是有效的 Win32 应用程序。
这次 stackoverflow拯救了我。按照高赞回答说的，修改掉一个文件名，顺便直接把系统标成 win 后，这位大爷终于不报错了（您这个库完全是按照 Linux 的文件系统写的，根本没有照顾过 win 的情绪好不好）。去跑了 mujoco 的示例模型，能跑出来了。
收拾 gym
这时候我去运行题目， gym 又出问题了，
>>> import gym
>>> env = gym.make('Humanoid-v2')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\gym\envs\registration.py", line 167, in make
return registry.make(id)
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\gym\envs\registration.py", line 119, in make
env = spec.make()
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\gym\envs\registration.py", line 86, in make
env = cls(**self._kwargs)
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\gym\envs\mujoco\humanoid.py", line 12, in __init__
mujoco_env.MujocoEnv.__init__(self, 'humanoid.xml', 5)
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\gym\envs\mujoco\mujoco_env.py", line 27, in __init__
self.model = mujoco_py.load_model_from_path(fullpath)
AttributeError: module 'mujoco_py' has no attribute 'load_model_from_path'
还好 github 上有两个 issue 244, 192，说使用python 3.6, mujoco 0.5.7 with the mjpro131 package installed and gym 0.9.1这一整套配置就可以了。我惊喜地发现我默认安装了gym 0.10.5，我改成pip install gym==0.9.1，终于 work 了……
现在我已经没有耐心写作业了，手动再见……
下午更新
我想用 jupyter 写作业，发现那边虚拟环境还要配置。参考了简书文章的后半部分，
# 安装ipykernel 库
conda install -n cs294hw1 ipykernel
# （在虚拟环境中）注册环境
python -m ipykernel install --user
应该是我遗漏了参数，这个环境被命名为 python3 了，我去C:\Users\shens\AppData\Roaming\jupyter\kernels\python3中找 json 文件改了一下。好啦开始写作业。
