SwiftQ 用于服务器端Swift应用程序的分布式任务队列

SwiftQ 用于服务器端Swift应用程序的分布式任务队列

SwiftQ

What is it?

SwiftQ is a distributed task queue for server side swift applications. Task queues are used as a mechanism to distribute work across machines. SwiftQ uses messages to communicate between clients and workers. In this case the message broker is Redis. SwiftQ uses the reliable queue pattern. This ensures that all tasks get processed even in the event of networking problems or consumer crashes. SwiftQ can be used for real time operations as well as delayed execution of tasks. SwiftQ can consist of multiple producers and consumers allowing for high availability and horizontal scaling.

Uses

SwiftQ is made to execute tasks outside of the usual HTTP request-response cycle. Examples of this are

Sending emailsRecording metricsDeleting/deactivating dataTalking with a third-party APISpreading out a large numbers of database inserts over timeProcessing imagesCreating thumbnailsAnything that takes more than 200ms to accomplish

Features

Scheduling tasksPeriodic tasksChaining tasksTask dependency injectionRecovery strategies for task failureAt-least-once delivery

Configuration

Redis Configuration

It is recommended that a separate Redis database is used to avoid conflicting name space.

let redisConfig = RedisConfig(redisDB: 0, hostname: "127.0.0.1", port: 6379, password: nil)

let configuration = Configuration(pollingInterval: 1000, enableScheduling: true, concurrency: 4, redisConfig: .development, tasks: [EmailTask.self])

For convenience during development RedisConfig has a static variable development that returns a RedisConfig with the default hostname and port. pollingInterval is the interval in milliseconds that SwiftQ will poll the scheduled queue. enableScheduling can be set to false in order to disable a consumer from polling the scheduled queue. Note polling is only used for monitoring the scheduled queue which uses Redis's Sorted set type. For all other tasks Redis's BRPOPLPUSH command is used to determine when tasks are available to be executed. All tasks must be registered in order for SwiftQ to process them.

Usage

Producer

Defining a producer and pushing a task onto the task queue.

let producer = try SwiftQProducer(redisConfig: .development)let demoTask = DemoTask()try producer.enqueue(demoTask)

Consumer

Creating a consumer. As soon as the consumer is started it will start processing tasks. The consumer must be initialized with the types of tasks you want to consume.

let consumer = try SwiftQConsumer(configuration)consumer.start()

Note a single app can be both a consumer and a producer, this may make deployment easier than having machines that are dedicated to being consumers.

Creating tasks

Every task must conform to the Task protocol. The storage property is there to allow SwiftQ to store extra information on your model, things like id, number of execution attempts etc. Tasks automatically conform to Codable. If you need to provide custom encoding logic include func encode(to encoder: Encoder) throws in your task.

final class EmailTask: Task { let storage: Storage let email: String init(email: String) { self.storage = Storage(EmailTask.self) self.email = email } func execute() throws { }}

Scheduling tasks

Scheduling is handy for things like sending follow up emails or sending reminders. Any task is schedulable.

try producer.enqueue(task: demo, time: .seconds(30))

This task will run in 30 seconds.

NOTE: The task does not fire exactly at the time supplied. Rather, once that time has passed, the task moves from the scheduled queue to the work queue and will be completed as workers are free to process it.

Periodic tasks

Periodic tasks have many uses. For example a web application could poll an API every 10 minutes to collect data. SwiftQ would handle invoking code to call the API, process the results and store them in a persistent database for later use by a client.

final class PollTask: PeriodicTask { let storage: Storage let url: String init(url: String) { self.storage = Storage(PollTask.self) self.url = url } func execute() throws { // Make request } var frequency: PeriodicTime { return .daily(minute: 30, hour: 5) }}

This task will run at 5:30 am every day. Periodic tasks must conform to the PeriodicTask protocol.

Advanced Usage

By default all consumers consume tasks from the same queue. You may want to specify a custom queue which only certain tasks are routed to. To do this just add this to your task

var queue: String { return "custom"}

You will also need to specify the custom queue in the consumer configuration.

Note: Consumers can only consume from one queue.

Supported Types

SwiftQ encodes tasks to JSON before sending them to the broker. Therefore only types with native JSON representation can be supported. Supported types are:

StringNumberArrayBoolean

JSON Format

{ "storage" : { "retryCount" : 0, "taskType" : "task", "name" : "EmailTask", "enqueuedAt" : 1507815271, "uuid" : "90872C7C-FCC8-4130-9872-87C619489664" }, "email" : "example@example.com"}

Installing

Update your Package.swift file with

.Package(url: "https://github.com/John-Connolly/SwiftQ.git", majorVersion: 0)

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。