Azure Storage Studying Notes

siwei • 2021-10-31 04:48 • 好物分享

Azure Storage is a Microsoft-managed cloud service that provides storage that is highly available, secure, durable, scalable and redundant. Whether it is images, audio, video, logs, configuration files, or sensor data from an IoT array, data needs to be stored in a way that can be easily accessible for analysis purposes, and Azure Storage provides options for each one of these possible use cases.

Within Azure there are two types of storage accounts, four types of storage, four levels of data redundancy and three tiers for storing files.

Storage Accounts

An Azure storage account contains all of your Azure Storage data objects: blobs, file shares, queues, tables, and disks. The storage account provides a unique namespace for your Azure Storage data that's accessible from anywhere in the world over HTTP or HTTPS. Data in your storage account is durable and highly available, secure, and massively scalable.

Azure Document : How to create a storage account.

A storage account provides a unique namespace in Azure for your data. Every object that you store in Azure Storage has an address that includes your unique account name. The combination of the account name and the Azure Storage service endpoint forms the endpoints for your storage account.

The Azure Storage platform comprises the following data services:

Azure Blobs are an immensely scalable object store for text and binary data. Also includes support for big data analytics through Data Lake Storage Gen2.
Azure Files are organized file shares for cloud or on-premises deployments.
Azure Queue is a messaging store for consistent messaging between application components.
Azure Tables are NoSQL store for schema-less storage of structured data.
Azure Disks are block-level storage volumes for Azure Virtual Machines.

Type of Storage Accounts:

Standard general-purpose v2
Premium block blobs
Premium file shares
Premium page blobs

Note: https://cloud.netapp.com/blog/azure-anf-blg-azure-storage-limits-at-a-glance

Resource	Regions	Limit per Storage Account
Number of storage accounts per region	N/A	250
Maximum storage capacity	All	5 PB
Maximum request rate	All	20,000 requests per second
Maximum ingress	US and Europe	10 Gbps
Maximum ingress	Other Regions	5 Gbps**
Maximum egress	US	50 Gbps*
Maximum egress	Other Regions	10 Gbps**
Maximum number of virtual network rules	All	200
Maximum number of IP address rules	All

Container

A container organizes a set of blobs, similar to a directory in a file system. A storage account can include an unlimited number of containers, and a container can store an unlimited number of blobs.

Azure Blob Storage

Blob storage includes three types of resources explained below:

The following points describe the use case scenarios:

Serving images or documents directly to a browser
Storing Files for distributed access
Streaming video and audio
Writing to log Files
Storing data for backup, restore, disaster recovery and archiving
Storing data for analysis by an on-premises or Azure-hosted service

Azure storage offers various access tiers, which allows storing the blob object data in a very cost-effective manner. The available access tiers include:

Hot– Augmented for storing frequently accessed data.
Cool– Optimized for storing less frequently accessed data, and the storage period lasts for at least 30 days.
Archive– Enhanced for storing rarely accessed data and the storage period lasts for at least 180 days with flexible latency requirements

Speed Performance:

A single blob supports up to 500 requests per second. If you have multiple clients that need to read the same blob and you might exceed this limit, then consider using a block blob storage account. A block blob storage account provides a higher request rate, or I/O operations per second (IOPS).

You can also use a content delivery network (CDN) such as Azure CDN to distribute operations on the blob. For more information about Azure CDN, see Azure CDN overview.

Limits for Blob storage:

Resource	Type of Blob	Limit
Maximum size of single blob container		5 PB
Maximum number of blocks	Block / Append	50,000
Maximum block size	Block	100 MB
Maximum total block size	Append	4 MiB
Maximum total blob size	Block	Approx. 4.75 TB
Maximum total blob size	Append	Approximately 195 GB
Maximum total blob size	Page	8 TB
Maximum stored access policies per container	All	5
Target request rate for blob	All	500/second
Target throughput for blob	Page	60 MB/s
Target throughput for blob	Block	Depends on storage account ingress/egress limits

Azure Storage Files

Azure Files provide fully managed File shares in the cloud that are approachable via the industry-standard SMB. Azure File shares can be attached parallelly by cloud or on-premises deployments of Windows, Linux, and macOS. It can be cached on Windows servers with Azure File Sync for quicker access. It permits the user to set up highly obtainable network file shares that can be accessed by using the standard Server Message Block (SMB) protocol. Multiple VMs can share similar files with both read and write permissions.

The only contrast between Azure Files and files on a corporate file share is, user can access the files from anywhere by using a URL that points to the file and contains a shared access signature (SAS) token. SAS tokens can be generated by the user; they allow specified access to a private asset for a specific time period.

Storage tiers

Azure Files offers four different tiers of storage, premium, transaction optimized, hot, and cool to allow you to tailor your shares to the performance and price requirements of your scenario:

Premium: Premium file shares are backed by solid-state drives (SSDs) and provide consistent high performance and low latency, within single-digit milliseconds for most IO operations, for IO-intensive workloads. Premium file shares are suitable for a wide variety of workloads like databases, web site hosting, and development environments. Premium file shares can be used with both Server Message Block (SMB) and Network File System (NFS) protocols.
Transaction optimized: Transaction optimized file shares enable transaction heavy workloads that don't need the latency offered by premium file shares. Transaction optimized file shares are offered on the standard storage hardware backed by hard disk drives (HDDs). Transaction optimized has historically been called "standard", however this refers to the storage media type rather than the tier itself (the hot and cool are also "standard" tiers, because they are on standard storage hardware).
Hot: Hot file shares offer storage optimized for general purpose file sharing scenarios such as team shares. Hot file shares are offered on the standard storage hardware backed by HDDs.
Cool: Cool file shares offer cost-efficient storage optimized for online archive storage scenarios. Cool file shares are offered on the standard storage hardware backed by HDDs.

Limitations related to file shares

Resource	Standard file shares	Premium file shares
Minimum share size	N/A	100 GB
Maximum share size	5 TB by default, can be increased up to 100TB	100 TB
Maximum file size	1 TB	4 TB
Maximum IOPS	1,000 IOPS* - Can be increased to 10000 IOPS	100,000 IOPS
Maximum stored access policies**	5	5
Target throughput**	60 MB/sec** - can be increased to 300MB/S	Ingress 4,136 MB/s Egress 6,204 MB/s
Maximum number of share snapshots	200	200
Maximum directory/file name length (chars)	2,048	2,048
Maximum hard links	N/A	178

Azure Blob Storage vs File Storage

Azure Blob Storage and File Storage, both services have their own defined properties and are implemented in different scenarios. Azure Files provides fully managed and organized cloud file shares that can be accessed from anywhere. Azure Blob Storage permits the storage of unstructured data and it can be accessed at a massive scale

Consider a development environment where every developer needs access to IDE and tools without using the internet to download it. In this situation, Azure Blob Storage would meet the need and using which the developer can only store development tools then give a link to the team to access the Blob location.

For implementing a File server in an organization, the user should choose the Azure Files option. A File server is used to share Files across departments in an organization. When it comes to File sharing, end user should not be allowed to access the copies of the file from its URI and need to be mapped locally in the computers. This is when Azure File Storage fits the organization’s need.

Azure Storage Queues

Azure Queue storage is an Azure service that implements cloud-based queues. Each queue maintains an inventory of messages. Application components access a queue employing a REST API or an Azure-supplied client library. Typically, you’ll have one or more sender components and one or more receiver components. Sender components add messages to the queue. Messages are retrieved from the front of the queue for processing by receiver components. The subsequent illustration shows multiple sender applications adding messages to the Azure Queue and one receiver application retrieving the messages. Storage Queues are part of the Azure Storage infrastructure, feature a simple REST-based GET/PUT/PEEK interface, providing reliable, persistent messaging within and between services.

Concepts of Queue service

User Case:

Imagine the user works as a developer for a major news organization that reports breaking news alerts. The company employs a worldwide network of journalists that are constantly sending updates through a web portal and a mobile app. A middle tier web service layer then takes those alert updates and publishes them online through several channels. However, it’s been noticed the system is missing alerts when globally significant events occur.

The middle tier provides plenty of capacity to handle normal loads. However, a look at the server logs revealed the system was overloaded when several journalists tried to upload larger breaking stories at the same time. Some writers complained the portal became unresponsive, and others said they lost their stories altogether. The user has spotted a direct correlation between the reported issues and the spike in demand on the middle tier servers.

Clearly, user needs a way to handle these unexpected peaks. At such a situation user doesn’t want to add more instances of the website and middle tier web service because they are expensive and, under normal conditions, redundant. They could dynamically spin up instances, but this takes time and they would have the issue waiting for new servers to come online.

This problem can be solved by using Azure Queue storage. A storage queue is a high-performance message buffer that can act as a broker between the front-end components and the middle tier. The front-end components place a message for each new alert into a queue. The middle tier then retrieves these messages one at a time from the queue for processing. At times of high demand, the queue may grow in length, but no stories will be lost, and the application will remain responsive. When demand drops back to normal levels, the web service will catch up by working through the queue backlog.

Azure Table Storage

Azure Table storage behaves as a service that stores structured NoSQL data inside the cloud, producing an attribute store with a schema less design. Because Table storage is schema less, it is easy to adapt your data because the needs of your application evolve. Access to Table storage data is fast and cost-effective for several sorts of applications and is usually lower in cost than traditional SQL for similar volumes of knowledge.

Azure Table Storage Use case

Table storage is employed to store flexible datasets like user data for web applications, address books, device information, or other metadata that the service requires. User can store any number of entities in a table, and a storage account may contain any number of tables, up to the capacity limit of the storage account Azure Table Storage Pricing. Azure tables are perfect for storing structured, non-relational data. Real time uses of Table storage include:

Storing datasets that do not require complex joins, foreign keys, or stored procedures and may be denormalized for fast access
Quickly querying data using a clustered index
Accessing data using the OData protocol and LINQ queries with WCF Data Service .NET Libraries

Azure Disk Storage

Azure managed disks are block-level storage parts that are managed by Azure and used with Azure Virtual Machines. Managed disks are similar a physical disk in an on-premises server but virtualized. In managed disks, all you must do is specify the disk size, type, and provision the disk. Once the disk is provisioned, Azure handles the rest. Each disk can take one of three roles in a virtual machine:

OS disk. One disk in each virtual machine contains the operating system files. When user creates a virtual machine, he/she selects a virtual machine image and that fixes the operating system and the OS disk that’s attached to the new machine. The OS disk has a maximum capacity of 2,048 GB.
Data disk. User can add one or more data virtual disks to each virtual machine to store data. For example, database files, website static content, or custom application code should be stored on data disks. The number of data disks that can be added depends on the virtual machine size. Each data disk has a maximum capacity of 32,767 GB.
Temporary disk. Each virtual machine contains a single temporary disk, which is used for short-term storage applications such as page files and swap files. The contents of temporary disks are lost during maintenance events, so do not use these disks for critical data. These disks are local to the server and are not stored in a storage account.

Azure Disk Storage Use case

The user manages a healthcare organization, and he is beginning a lift-and-shift migration to the cloud where many of their systems will be running on Azure virtual machines. These systems have a variety of usage and performance profiles which are highly confidential. The user is concerned about the storage and does not want to access that data outside the virtual machine.

To address these needs, the organization’s got to option is Azure Disk Storage. The Azure Disk Storage is capable of,

“Lift and shift” of applications that use native file system APIs to read and write data to persistent disks.
Preserve data that is not required to be accessed from outside the virtual machine to which the disk is attached.

References

版权声明：
作者：siwei
链接：https://www.techfm.club/p/5365.html
来源：TechFM
文章版权归作者所有，未经允许请勿转载。

THE END

二维码

常见的算法题目

< <上一篇

罗翔：领导的看法

下一篇>>

搜索内容