Tens of thousands of companies have saved petabite -scale data in Amazon Elastic File System (Amazon EFS), and most of them are data for applications that use EFS to container.The Amazon EFS file system can be connected to the Amazon Elastic Container Service (ECS) and Elastic Kubernetes Service (EKS).Amazon EFS is a perfect option for container storage, as it is a full -managed service that is easy to set when adding or deleting data, like a container frustration.In addition, it can be expanded not only to petite -class data, but also to total gigabytes per second and thousands of IOPS.This blog introduces some frequently asked questions about the best practices for using EFS in containered applications.
In general, shared file storage is suitable for long -term executing containers that require recovery for disability and containers that need to share data between each other.Here are some examples you often see.
Another option that permanently converses container data is Amazon Elastic Block Storage (EBS).This is often suitable when it is not necessary to share data with other containers, such as a distributed system database such as MySQL or Kafka or Cassandra.
In most cases, you can create a new file system for each new application that needs to share or hold data.This can protect and manage all your application data in one place.This has two exceptions.The first is the case where the applications that include the data required for the application already exist, and instead of creating a new file system, only the existing file system is connected to the container.The second is that the container requires only the readable data, and the file system is mounged only for reading in both ECS and EKS.
Another reason for sharing the file system is the scale.Each file system to create at least one mount target can create up to 400 mount targets for each Amazon Virtual Private Cloud (VPC).If there are more than 400 applications on VPCs and each needs a unique shared file storage, one file system is divided into multiple directors.Even if there are less than 400 applications, the throughput of the file system is based on the total amount of stored data, so it is easier to manage the file system between multiple containers, and the burst mode file system.The total throughput may be higher.To do this, create a file system and create a directory for each unique application, such as /Myapplication1 and /MyApplication2.If the source directory is specified as /myapplication1 when mounted a file system to a container, the container will be route to the location and will not be able to display other application data.From a security point of view, if the administrator who starts the application can trust, this solution works optimally.Administrators are responsible for scapping each application only for directory containing data.
Both ECS and EKS launch the container in multiple Availability zones.To make the application access to the EFS wherever the application is started, we recommend that you create her EFS mount target in any of your current regions.This is done by default when creating a file system using the EFS console.There is no new additional charge to create additional mount targets for the file system.
Set the container for the file system using the name of the DNS, File-System-ID.EFS.AWS-Region.Amazonaws.We recommend that you use COM format.With this DNS name, the lookup is automatically solved to the same Availability zone mount target as the application, so the cost and performance of the network are optimized.This is automatically performed with services and frameworks that integrate EFS and native, such as ECS and EKS.
Both the EFS data and the data you save can be encrypted.You can enable saving data encryption when creating a file system using a customer master key (CMK) managed by AWS or customer.The encryption of the data during the transfer is set for each connection.To encrypt the data being transferred, first make sure that the EFS mount helper is installed on the container host you are using, and to mount it with the "-O TLS" option.
The EFS file system has two performance modes, general -purpose and maximum I/O.Normally, general -purpose performance mode is the most suitable for interactive applications that benefit from shortening the latency for each operation, such as content management systems, developer tools, and data science notebook books.The maximum I/O performance mode file system is ideal for analyzing, machine learning training, or other workloads that perform parallel operations from hundreds or thousands of containers.Searching for the highest consolidation throughput and IOPS as much as possible, it is not affected by the latency of individual operations.
Customers who use EFS for container storage usually use Provisioned ThroughPut to receive consistent services for end users.Bursting ThroughPut file system throughput may not be enough throughput to newly provided applications to based on the amount of stored data.To adjust this, configure Provisioned ThroughPut in the exact throughput that ends users need.For example, the Jenkins file system is usually set with 50 to 150 mib/s Provisioned ThroughPut, and the Nexus or Artifactory Repository System is set at 512-1024 MIB/S.
The good thing about PROVISIONED THROUGHPUT is that it can only be paid to the amount of throughput that exceeds the base line rate received by Bursting ThroughPut, taking into account the amount of saved data.If the throughput is higher than the provisioning, the high throughput of the two will be provided.For example, if a Jenkins container is provided to a file system with a 50 mib / s Provisioned ThroughPut, as the total storage approaches his 1 tib, the payment to the sloup will gradually decrease.After the PROVISIONED THROUGHPUT payment is gone, the allowed throughput will be paid based on 50 MIB/s per TIB saved on the slide system.End users can enjoy consistent services because the throughput is less than 50 mib/s, even if the storage falls below 1 tib.
of course.EFS Low -frequency access (IA) storage class can reduce the price to save data in EFS up to 92 %.To use this, set the LifeCycle Management in the file system and specify the period for a file that has not been read or write to the IA.This period is a minimum of 14 days, up to 90 days.For example, artifacts, such as Nexus and Artifactory, are usually accessible only to the latest build artifacts, so 80 % or more of data can be shifted to IA.As a result, GB per month based on the eastern US (northern Virginia) (US-EAST-1) region..08 USD mixture costs are incurred.
With AWS Backup, you can easily back up all data file systems in container -conated applications.To start, access the AWS Backup console and set a backup plan, and then set the backup plan so that the resource is assigned with a tag such as "K: backup, V: DailyBackup".Next, when creating a container file system, add the "K: backup, V: DailyBackup" tag.AWS Backup automatically detects the file system and starts backup according to the set backup policy.If you need to accidentally delete the data and restore it from the backup, you can instruct AWS Backup to restore backup to any of the existing file systems or new file systems.
EFS outputs CloudWatch Metrics, which is useful for monitoring file systems.The sample dashboard is released.This includes a metrics formula and an alarm value for easily starting.In particular, please note the following.
The ECS document has a tutorial for starting the use of EFS with ECS.In the case of EKS, the start procedure is on the EFS CSI driver GitHub page.
I am glad if this guidance will help everyone who deploys containers that need shared storage.If you have any other questions that I can answer, please comment.