I’m running RL experiments using IsaacGym (and MuJoCo) on a headless Linux GPU server (Ubuntu 20.04, NVIDIA A100) via SSH from my MacBook. The server does not have a GUI installed, so I need to render the simulation output locally on my Mac.

I’ve tried the following approaches:

1. VNC with Xvfb: I set up Xvfb and x11vnc to create a virtual display and forward it to my Mac. I can connect with RealVNC Viewer and it would display plt and tkinter figure, but the display resolution and quality are poor. As for IsaacGym and MuJoCo, render window does not update properly.

   Here is an example:

def mujoco():
    env = gym.make('Ant-v3')
    obs = env.reset()
    print("Starting Mujoco environment rendering test...")
    
    for step in range(1000):
        env.render()
        
        action = env.action_space.sample()
        obs, reward, done, info = env.step(action)
        time.sleep(0.1)

        if done:
            obs = env.reset()

    env.close()

# -- output of trying to render the MuJoCo env --
...# Omit other outputs above
File "Anaconda3/envs/rl_env/lib/python3.8/site-packages/mujoco_py/opengl_context.pyx", line 48, in mujoco_py.cymj.GlfwContext.__init__
    self.window = self._create_window(offscreen, quiet=quiet)
File "Anaconda3/envs/rl_env/lib/python3.8/site-packages/mujoco_py/opengl_context.pyx", line 99, in mujoco_py.cymj.GlfwContext._create_window
    raise GlfwError("Failed to create GLFW window")
mujoco_py.cymj.GlfwError: Failed to create GLFW window

2. VirtualGL: I attempted to use VirtualGL but encountered issues installing it on my server.

Has anyone encountered this issue? What is the best approach to render IsaacGym (or MuJoCo) output from a headless Linux GPU server to a MacBook? Thanks~!

Settings:

• Local: M2, Sequoia 15.3.2
• Remote: Ubuntu Linux, 20.04
• Python version: 3.8