CouchDB

我构建了一个基于CouchDB的移动应用程序。

出于安全原因，我必须确保文档只能由允许执行此操作的用户读取。 由于我无法在文档级别管理访问权限，因此我为每个用户创建一个couchdb数据库，并使用筛选复制从每个用户数据库中的主couchDB数据库复制文档。

这个模型工作得很好，但今天我遇到了巨大的性能问题。

我尝试将所有复制连续，过滤和双向复制，但在80个用户（因此81个数据库和160个同时连续复制）之后，复制过多，我的couchDB服务开始减速甚至有时崩溃。 注意到所有数据库都在同一台服务器上（我不能有多台服务器）

我试图实现“手动”复制，但即使这样，当我需要将文档从我的主数据库复制到我的所有80个用户数据库时，每个从主数据库到用户数据库的过滤复制大约需要30秒。

也许我的复制过滤器有问题，我为每个文档存储允许查看它的用户列表。 由于每个用户都拥有自己的数据库，因此我只复制允许用户在其数据库中查看的文档。 这是我的复制功能：

我的应用程序的目标是获得大约1000个用户，这对于当前的架构/性能来说是完全不可能的。

我有三个问题：1。即使我认为这是不可能的，是否有可能在同一台服务器上连续复制大约1000个数据库？ 2.我的复制过滤器有什么问题吗？ 有没有办法改进它以实现快速数据库复制？ 3.如果当前的架构一点都不好，那么你会在我的案例中提出什么样的架构？

非常感谢你 ！

I built a mobile application based on CouchDB.

For security reason, i have to make sure that a document can be read only by the users allowed to do do it. As i cannot manage the access right at document level, i create one couchdb database per user, and i replicate documents from my main couchDB database in each user database with a filtered replication.

This model work very well, but today i faced huge performance issues.

I tried to have all my replications continuous, filtered and bi-directionnal, but after 80 users (so 81 databases and 160 simultanous continuous replications), there was too much replications and my couchDB service start to slow down and even crashed sometimes. Notices that all the databases are on the same server (and i could not have more than one server)

I tried to put in place a "manual" replications, but even this way when i need to replicate a document from my main database to all my 80 users databases, each filtered replication from my main database to a user database take around 30 seconds.

Maybe i have an issue with my replication filter, i store for each document a list of users allowed to see it. As each user has it own database, i replicate only the document the user is allowed to see in its database. Here is my replication function :

The goal of my application is to get around 1000 users, that is totally impossible with the current architecture / performance.

I have three questions : 1. Even if i think that it's not possible, Is it possible to get about 1000 databases in continuous replication on the same server? 2. Is there anything wrong with my replication filter? Is there any way to improve it to have fast databases replications? 3. If the current architecture is not good at all, what kind of architecture would you advise in my case?

Thank you very much !

步骤0总是识别瓶颈。 我根据您的场景概述我的第一个猜测是查看I / O性能。 查看

和

每个数据库都有自己的读写文件描述符，因此服务器上打开的数据库数量会增加，所需的I / O资源也会增加。 希望这可以帮助

We finally changed our global project architecture. The main server cannot handle more than 100 replicated databases even if the configuration limits can be changed, after 80 synchronied databases couchdb logs start to explode. I may wrong, but i think that this kind of architecture is not possible on a single server.

Here is the solution we put in place. We removed all the users databases and we plugged all our mobile applications directly on the main database and do a filtered replication directly on the main database : .html#replication by using this solution : Example 3: filter function inside of a design document

This new model is now working we did some stress tests and we didn't get any issue until 1000 simultaneous users.

Just be aware that pouchDB, to replicate a database, ask couchdb all the modifications applied on the main database since the last synchronisation (even for filtered replication). So when you create a new pouchdb database and synchronise it, if your main couchDB is old and has a big historical (check couchdb _changes API), it can take a very (very) long time !