I am aware that there is a GitHub repository for using a Dark Mode Task Dialog. THis is the sample from that webpage:

It uses Microsoft Detours, which is no longer available as a NuGet Package and has to be manually downloaded / compiled.

So I am trying to use the "Undocumented Route". I confess that I have had discussions with ChatGPT and DeepSeek, and I have also done alot of Googling. But I have come stuck.

MY application class is configured for dark mode:

// Declare function for setting preferred app mode
typedef enum _PREFERRED_APP_MODE {
    Default,
    AllowDark,
    ForceDark,
    ForceLight,
    Max
} PREFERRED_APP_MODE;

typedef BOOL(WINAPI* fnAllowDarkModeForWindow)(HWND, BOOL);
typedef PREFERRED_APP_MODE(WINAPI* fnSetPreferredAppMode)(PREFERRED_APP_MODE appMode);

    static void EnableDarkModeForTaskDialog(HWND hWnd)
    {
        HMODULE hUxtheme = LoadLibraryEx(L"uxtheme.dll", nullptr, LOAD_LIBRARY_SEARCH_SYSTEM32);
        if (hUxtheme)
        {
            auto pSetPreferredAppMode = (fnSetPreferredAppMode)GetProcAddress(hUxtheme, MAKEINTRESOURCEA(135)); // undocumented
            auto pAllowDarkModeForWindow = (fnAllowDarkModeForWindow)GetProcAddress(hUxtheme, MAKEINTRESOURCEA(133)); // undocumented

            if (pSetPreferredAppMode)
                pSetPreferredAppMode(AllowDark);

            if (pAllowDarkModeForWindow)
                pAllowDarkModeForWindow(hWnd, true);

            FreeLibrary(hUxtheme);
        }
    }

In my app class I override DoMessageBox:

int CTestDarkModeApp::DoMessageBox(LPCTSTR lpszPrompt, UINT nType, UINT nIDPrompt)
{
    // Define the task dialog configuration
    TASKDIALOGCONFIG tdc = { sizeof(TASKDIALOGCONFIG) };
    tdc.hwndParent = AfxGetMainWnd()->GetSafeHwnd();
    tdc.dwFlags = TDF_ALLOW_DIALOG_CANCELLATION | TDF_SIZE_TO_CONTENT;
    tdc.dwCommonButtons = 0;
    tdc.pszWindowTitle = L"Test Dark Dialog";
    tdc.pszContent = lpszPrompt;

    // Set dialog width dynamically
    tdc.cxWidth = DetectMessageBoxWidth(); // Equivalent to SetDialogWidth()

    // Set callback function
    tdc.pfCallback = [](HWND hWnd, UINT msg, WPARAM wParam, LPARAM lParam, LONG_PTR lpRefData) -> HRESULT {
        if (msg == TDN_DIALOG_CONSTRUCTED)
        {
            // Enable dark mode for the TaskDialog
            CTestDarkModeApp::AllowDarkModeForWindow(hWnd, true);
        }
        return S_OK;
        };

    // Set dialog icons
    switch (nType & MB_ICONMASK)
    {
    case MB_ICONERROR:
        tdc.pszMainIcon = TD_ERROR_ICON;
        break;
    case MB_ICONWARNING:
        tdc.pszMainIcon = TD_WARNING_ICON;
        break;
    case MB_ICONINFORMATION:
        tdc.pszMainIcon = TD_INFORMATION_ICON;
        break;
    case MB_ICONQUESTION:
        tdc.pszMainIcon = TD_INFORMATION_ICON; // TaskDialogIndirect doesn't have a built-in question icon
        break;
    default:
        tdc.pszMainIcon = nullptr;
        break;
    }

    // Set buttons based on the message box type
    switch (nType & MB_TYPEMASK)
    {
    case MB_YESNOCANCEL:
        tdc.dwCommonButtons = TDCBF_YES_BUTTON | TDCBF_NO_BUTTON | TDCBF_CANCEL_BUTTON;
        break;
    case MB_YESNO:
        tdc.dwCommonButtons = TDCBF_YES_BUTTON | TDCBF_NO_BUTTON;
        break;
    case MB_RETRYCANCEL:
        tdc.dwCommonButtons = TDCBF_RETRY_BUTTON | TDCBF_CANCEL_BUTTON;
        break;
    case MB_OKCANCEL:
        tdc.dwCommonButtons = TDCBF_OK_BUTTON | TDCBF_CANCEL_BUTTON;
        break;
    case MB_OK:
    default:
        tdc.dwCommonButtons = TDCBF_OK_BUTTON;
        break;
    }

    int nButtonPressed = 0;
    TaskDialogIndirect(&tdc, &nButtonPressed, nullptr, nullptr);

    return nButtonPressed;
}

The bulk of this code is my own from my existing application. But, I was using CTaskDialog. AI helped me adapt it to use TaskDialogIndirect and take advantage of the callback mechanism.

So, if I use this code:

AfxMessageBox(L"Message Box", MB_OK|MB_ICONINFORMATION);

The result is:

I do have the dark title bar, so that is something. But the rest of the dialog is still light. I thought it might have been possible to do this with the undocumentated functions. I can get many controls rendered in dark mode for regular CDialog with SetWindowTheme.

Update

I have made significant progress, thanks to the comments provided about using the SetWindowsHookExW / UnhookWindowsHookEx approach.

Here is my CBTProc:

inline LRESULT CALLBACK CBTProc(int nCode, WPARAM wParam, LPARAM lParam)
{
    if (nCode == HCBT_CREATEWND)
    {
        CBT_CREATEWND* pccw = reinterpret_cast<CBT_CREATEWND*>(lParam);

        if (pccw->lpcs->lpszClass == WC_DIALOG)
        {
            AllowDarkModeForWindow((HWND)wParam, TRUE);
            SetWindowTheme((HWND)wParam, L"DarkMode_Explorer", 0);
        }
    }

    return CallNextHookEx(0, nCode, wParam, lParam);
}

Then, my original code I provided is tweaked (similar to the gist):

DarkModeTools::SetPreferredAppMode();
HHOOK hhk = SetWindowsHookEx(WH_CBT, DarkModeTools::CBTProc, 0, GetCurrentThreadId());
TaskDialogIndirect(&tdc, &nButtonPressed, nullptr, nullptr);
if (hhk) UnhookWindowsHookEx(hhk);

When I run it:

Two main issues:

1. Text is black
2. Command buttons at the bottom are not in dark mode.

Also, when I use a more complicated task dialog, like this:

Four main issues:

1. Text is black
2. Command buttons at the bottom are not in dark mode.
3. Show / Hide details.
4. Bottom line

Update 2

Some improvement if I comment out if (pccw->lpcs->lpszClass == WC_DIALOG) line:

Buttons are now coloured. It feels like I need to be able to process a OnCtlColor handler for the task dialog. That might fix he other issues.

Update 3

I tried a second hook:

    static LRESULT CALLBACK CallWndProcHook(int nCode, WPARAM wParam, LPARAM lParam) {
        if (nCode >= 0) {
            CWPSTRUCT* pMsg = (CWPSTRUCT*)lParam;
            if (pMsg->message == WM_CTLCOLOR) {
                // WM_CTLCOLOR detected
                // pMsg->hwnd is the handle to the control
                // pMsg->wParam is the HDC of the control
                // pMsg->lParam is the handle to the child window
                HDC hdc = (HDC)pMsg->wParam;
                SetBkColor(hdc, RGB(255, 255, 255)); // Set the background color to white
                SetTextColor(hdc, RGB(0, 0, 0)); // Set the text color to black
                return (LRESULT)GetStockObject(NULL_BRUSH); // Return a null brush to prevent background painting
            }
        }
        return CallNextHookEx(NULL, nCode, wParam, lParam);
    }

and adjusted the main code:

    HHOOK hhk = SetWindowsHookEx(WH_CBT, DarkModeTools::CBTProc, 0, GetCurrentThreadId());
    HHOOK hhk2 = SetWindowsHookEx(WH_CALLWNDPROC, DarkModeTools::CallWndProcHook, NULL, GetCurrentThreadId());

    TaskDialogIndirect(&tdc, &nButtonPressed, nullptr, nullptr);
    if (hhk) UnhookWindowsHookEx(hhk);
    if (hhk2) UnhookWindowsHookEx(hhk2);

But the code is never intercepted sadly.