最新消息:雨落星辰是一个专注网站SEO优化、网站SEO诊断、搜索引擎研究、网络营销推广、网站策划运营及站长类的自媒体原创博客

如何绕过 Amazon Aurora 1MB 数据 API 限制?

网站源码admin29浏览0评论

如何绕过 Amazon Aurora 1MB 数据 API 限制?

如何绕过 Amazon Aurora 1MB 数据 API 限制?

我正在努力从 Amazon Aurora Postgres DB 检索商店列表,我需要完整的列表。

我有超过 20.000 家商店,但由于 Amazon Aurora 数据 API 的 1MB 限制而失败..

我收到的消息:


{
    "error": "INTERNAL_SERVER_ERROR",
    "message": "select \"e0\".*, \"e1\".\"id\" as \"location_id\" from \"shop\" as \"e0\" left join \"location\" as \"e1\" on \"e0\".\"id\" = \"e1\".\"shop_id\" where \"e0\".\"name\" is not null and \"e0\".\"online\" = false and \"e0\".\"status\" = 'active' and \"e1\".\"formatted_address\" is not null and \"e1\".\"latitude\" is not null and \"e1\".\"longitude\" is not null order by \"e0\".\"uuid\" asc limit 10000 - Database returned more than the allowed response size limit"
}

I use mikro-orm and heres sample of my code:

    @Get()
    @Middleware(
        validateQueryParams(
            Joi.object({
                offset: Joi.number().integer().raw().min(0),
                limit: Joi.number().integer().raw().min(0).max(ShopRepository.MaxResults),
            })
        )
    )
    async list(req: Request, res: Response, next: NextFunction) {
        logger.trace('ShopLocationController::list() - Enter');

        const { limit, offset } = parseFixedLengthCollectionParams(req.query);

        // Connect to all jurisdictional databases and retrieve their shop locations
        const databases = Adapter.allJurisdictionalDataApis();
        const results: ShopSetResult[] = await Promise.all(
            databases.map(async database => {
                await database.connect();
                const shopRepository = database.em.getRepository(Shop) as ShopRepository;
                const [shops, total] = await shopRepository.findHavingLocation(0, limit);
                return { shops, total };
            })
        );
        const completeShopsInJurisdictionBlocks = results.map(
            (shopset: ShopSetResult) => shopset.shops
        );
        const completeShopSet = completeShopsInJurisdictionBlocks.flat();

        // Calculate the total by adding all the totals from all the jurisdictions together
        const total = results.reduce((runningTotal, shopset) => (runningTotal += shopset.total), 0);

        // Sort shops and apply offsets and limits
        completeShopSet.sort((a, b) => (a.uuid > b.uuid ? 1 : -1));
        const shopsAfterOffsetAndLimit = completeShopSet.slice(
            offset,
            limit ? offset + limit : undefined
        );

        const list = toCollection(shopsAfterOffsetAndLimit, total, offset);
        return jsonSuccess(res, list);
    }

任何人都可以根据我的代码提供示例以某种方式对其进行分页但最后返回完整列表吗?

回答如下:
发布评论

评论列表(0)

  1. 暂无评论