I have set up debugging, and using error_log("message"); works fine everywhere I have used it so far, except within an AJAX handler.

How does one log to the error handler within an AJAX handler?

E.g. The following is working and returning status 200, but not logging to the error log:

function my_action_handler() {
    // These are never logged
    error_log("my_action_handler was called"); //This is never logged
    $a = $_POST['a'];
    error_log("a:".$a);

    wp_send_json_success();
    die();
}
add_action( 'wp_ajax_my_action', 'my_action_handler' );