Opened 10 years ago

Last modified 10 years ago

#24 new enhancement

ror-farm next gen: lxfarm

Reported by: Vincent Caron Owned by: Lucas Bonnet
Priority: minor Component: RoR Farm
Keywords: Cc: Lucas Bonnet,


Front: HTTP/HTTPS load balancer and dispatcher

  • Simple mapping from a DNS list to a lxfarm instance (or instance group)
  • Optional "maintenance" page, on demand
  • Nginx OK

A lxfarm instance definition:

  • name
  • dimensions (RAM, processes/CPU, storage)
  • template

Lxfarm services:

  • MySQL: OK, quota via LVM (tested: OK). phpMyAdmin for admin, or a web-interface. MyISAM et InnoDB OK. User XtraDB 5.5 from Percona ?
  • Memcached: OK, any number per client, may choose its port+dimension

Networking, security:

  • We need one VLAN per client
  • MySQL: the API generates the passwords, the user does not (security guarantees)

Data model:

  • servermap: which server for which services,
    • list of available ressources
  • application: glue of N services
    • name
    • VLAN
    • DNS list (vhosts)
    • maintenance: on/off
    • service list (implicit)
  • service
    • application ID
    • type: lxfarm, mysql, memcache
    • dimensions: disk space, mem space, ...

Technical notes:

  • Needs specific locking for various service creation operations ?
  • Needs a database: prefer Perl/Python? over shell...

Change History (7)

comment:1 Changed 10 years ago by Vincent Caron

Owner: changed from Lucas Bonnet to Vincent Caron

comment:2 Changed 10 years ago by Vincent Caron


  • Write a simple SQL schema from those specs
  • Bootstrap first commands (app create, service create)

comment:3 Changed 10 years ago by Lucas Bonnet

Cc: Lucas Bonnet added

comment:4 Changed 10 years ago by Vincent Caron

Owner: changed from Vincent Caron to Lucas Bonnet

Commit hook was broken, I sent [590]. There's:

There is a very basic proof-of-concept lxf-app --create <name> and lxf-app --list.

Next step would be the implementation of the lxf-worker script (based on lxf-app) which is more or less the same thing as source:/ror-farm/ror-farm-add (create a LXC context for a given app, but also list, modify and delete it).

comment:5 Changed 10 years ago by Vincent Caron

Update on networking:

  • We won't use VLANs at first, since there is some setup and networking overhead and its not specifically needed: workers don't need to communicate between themselves, or if they do it should happen at a higher level, trhu the loadbalancer/dispatcher. We'll use simple ARP and IP filtering to isolate containers.
  • We'll assign one subnet per server, typically a 192.168.x.n where (because both Bearstech and EC2 use 10/8 internally) :
    • x = server ID
    • n = container ID
  • We can use static routes on N servers (where N<50 ?), then we might think BGP and bigger stuff. They can be easily generated: for each 'x' server, the static route for 192.168.x/24 goes to gw 192.168.x.1. The configuration is identical on all servers.
  • The 'veth' LXC stuff + Linux bridging is fine, we'll need a few iptables later. We need a simple policy to generate virtual MACs from assigned IPs.

comment:6 Changed 10 years ago by Vincent Caron

I removed the VLAN notion in [591] and added:

  • '' as primary key, it will be more useful than its name to map from and to its IP address
  • 'service': (server, local_id) as index. I expect to address a service with 192.168.<server>.<local_id> for instance...
  • 'service': an application might have several services of the same role, I added an 'instance' integer. We'll increment this number forever, this way we (and the user) will be able to distinguish all service instances, those running and those which have already been deleted. I removed the 'name' field, so we can name them by ourselves, internally. Eg: 3rd created MySQL database for app 'foobar' can be simply named 'foobar3'. The user will address this with a (appname, role, instance) triple: foobar, mysql, 3.

The 'local_id' should be IMHO a VARCHAR and encode the ressource name, it would be:

  • an IP (suffix) for a container (worker role)
  • a DB name for MySQL
  • ...

comment:7 Changed 10 years ago by cyberj

(In [595]) Lxc-farm : add python parser (see #24)

Note: See TracTickets for help on using tickets.