Using epoll() For Asynchronous Network Programming

Using epoll() For Asynchronous Network Programming

General way to implement tcp servers is “one thread/process per connection”. But on high loads this approach can be not so efficient and we need to use another patterns of connection handling. In this article I will describe how to implement tcp-server with synchronous connections handling using epoll() system call of Linux 2.6. kernel.

epoll is a new system call introduced in Linux 2.6. It is designed to replace the deprecated select (and also poll). Unlike these earlier system calls, which are O(n), epoll is an O(1) algorithm – this means that it scales well as the number of watched file descriptors increase. select uses a linear search through the list of watched file descriptors, which causes its O(n) behaviour, whereas epoll uses callbacks in the kernel file structure.

Another fundamental difference of epoll is that it can be used in an edge-triggered, as opposed to level-triggered, fashion. This means that you receive “hints” when the kernel believes the file descriptor has become ready for I/O, as opposed to being told “I/O can be carried out on this file descriptor”. This has a couple of minor advantages: kernel space doesn’t need to keep track of the state of the file descriptor, although it might just push that problem into user space, and user space programs can be more flexible (e.g. the readiness change notification can just be ignored).

To use epoll method you need to make following steps in your application:

Create specific file descriptor for epoll calls:  epfd = epoll_create(EPOLL_QUEUE_LEN);  where EPOLL_QUEUE_LEN is the maximum number of connection descriptors you expect to manage at one time. The return value is a file descriptor that will be used in epoll calls later. This descriptor can be closed with close()After first step you can add your descriptors to epoll with following call:  static struct epoll_event ev; int client_sock; ... ev.events = EPOLLIN | EPOLLPRI | EPOLLERR | EPOLLHUP; ev.data.fd = client_sock; int res = epoll_ctl(epfd, EPOLL_CTL_ADD, client_sock, &ev);  where ev is epoll event configuration sctucture, EPOLL_CTL_ADD – predefined command constant to add sockets to epoll. Detailed description of epoll_ctl flags can be found in epoll_ctl(2) man page. When client_sockWhen all your descriptors will be added to epoll, your process can idle and wait to something to do with epoll’ed sockets:  while (1) { // wait for something to do... int nfds = epoll_wait(epfd, events, MAX_EPOLL_EVENTS_PER_RUN, EPOLL_RUN_TIMEOUT); if (nfds < 0) die("Error in epoll_wait!"); // for each ready socket for(int i = 0; i < nfds; i++) { int fd = events[i].data.fd; handle_io_on_socket(fd); } }

Typical architecture of your application (networking part) is described below. This architecture allow almost unlimited scalability of your application on single and multi-processor systems:

Listener– thread that performs bind() and listen() calls and waits for incoming conncetions. Then new connection arrives, this thread can do accept() on listening socket an send accepted connection socket to one of theI/O-workers.I/O-Worker(s)– one or more threads to receive connections fromlistenerand to add them to epoll. Main loop of the genericI/O-workerlooks like last step of epoll using pattern described above.Data Processing Worker(s)– one or more threads to receive data from and send data toI/O-workersand to perform data processing.

As you can see, epoll()

If you want to read more about epoll or you want to look at some benchmarks, you can visit ​​epoll Scalability Web Page​​ at Sourceforge. Another interesting resources are:

​​The C10K problem​​: a most known page about handling many connections and various I/O paradigms including epoll().​​libevent​​: high-level event-handling library ontop of the epoll. This page contains some information about performance tests of epoll.

Related posts:

​​Nginx – Small, But Very Powerful and Efficient Web Server​​​​Monitoring nginx Server Statistics With rrdtool​​​​How To Get “Provider Independent” IP Address For Your Home Server?​​​​Looking For Optimal Solution: Benchmark Results Summary and Findings​​​​HAProxy – The Reliable, High Performance TCP/HTTP Load Balancer​​

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。