mycpen

Mycpen

记录学习历程与受益知识
github
telegram
bilibili

01_Reprint - Building an Image Hosting Service with Backblaze B2 + Cloudflare CDN

  1. BlazeB2 - Image hosting tool based on BackBlaze B2 and Cloudflare (not used)

Reprinted Articles#

  1. Backblaze B2 + Cloudflare CDN customizable domain 10G high-speed free external link image hosting

  2. Using Backblaze B2 + Cloudflare CDN + PicGo to achieve a customizable domain 10G free image hosting solution

  3. Using Backblaze B2 and Cloudflare as free image hosting

Thanks to ReCreyed for sharing. The following content is excerpted from the above 3 articles.

Article 1: Backblaze B2 + Cloudflare CDN#

All other articles on this blog use GitHub image hosting. I accidentally discovered the self-built image hosting solution of B2+CF+ShareX, and it can also customize the domain name, which led to this article. All images in this article come from this method of image hosting.

Image hosting refers to servers that store images. Website administrators will store images in dedicated server clusters for easier maintenance and management, and usually, a CDN is layered on top for distribution. There are many public image hosting services available, such as imgur, sm.ms, imgbox, etc. These image hosting services have opened APIs for developers to call. However, due to operational costs, many image hosting services have gone out of business or switched to paid services.

How to build your own image hosting? If you have a VPS server, you can install the Chevereto open-source image hosting website program. However, using a VPS solely for image hosting for blog external links is somewhat wasteful and not sustainable in the long run. It is best to store images on a large company's cloud service platform. Backblaze B2 is a cloud storage solution similar to Amazon AWS S3, but slightly cheaper. Backblaze's cloud storage offers each registered user 10G of free space and 1G of download traffic per day, with unlimited upload traffic. The prices after exceeding the free quota and comparisons with other mainstream providers are shown in the image below.

img

Storing 1T of content on Backblaze only costs $5/month, but the traffic fees are relatively expensive, and upload traffic is unlimited. Due to the Bandwidth Alliance, the exit from Backblaze to Cloudflare is completely free, so Backblaze B2 can be used with Cloudflare CDN to achieve unlimited free download traffic.

img

Backblaze announced a partnership with Cloudflare to waive the traffic fees from Backblaze to Cloudflare.

Thus, we get the following cloud storage/image hosting product:

  • 10G free storage space, the most reliable cloud storage service globally
  • Free upload
  • Free download
  • Customizable domain name
  • The strongest global CDN acceleration

Starting the Custom Domain Image Hosting Tutorial#

First, you need to have a domain name and choose a subdomain as the domain name for the image hosting. Regarding the naming of the secondary domain name for image hosting, there are generally the following options:

  • img
  • images
  • assets
  • static (not only stores images but also stores css, js)

Then, create a Backblaze B2 Bucket.

Register an account at Backblaze’s B2 storage, and after registration, select “Buckets” in the backend panel, then click “Create a Bucket” to create a storage bucket. Choose Public so that everyone can access it.

img

Then, return to “Buckets”, click on the bucket you just created, and click “Upload/Download”. Upload a temporary test file to the bucket, then click “i (information)” on the file you just uploaded. This is to obtain the information needed for the next step, which server your bucket is on.

img

img

In the screenshot above, you can see that my file is stored at https://f000.backblazeb2.com/ – remember this domain name. After obtaining the address, log in to Cloudflare, add a subdomain to your domain, and create a CNAME record pointing to the domain in the file address, for example, pointing img.coca.cc via CNAME to f000.backblazeb2.com, resulting in https://img.coca.cc/file/imgsgj/img.jpg pointing to https://f001.backblazeb2.com/file/imgsgj/img.jpg.

img

According to the record in the screenshot above, I use this subdomain img.coca.cc as the domain name for the image hosting, pointing to f000.backblazeb2.com. Ensure that the Cloudflare orange protection shield is enabled, which indicates that requests are routed through Cloudflare's CDN proxy layer. The default TTL for Cloudflare will be set to auto.

Although Cloudflare CDN is enabled, it is found that the cache never hits (MISS). We need to add a cache directive (default is not cached), open the bucket settings, and set the bucket information to {"cache-control":"max-age=43200"}, where 43200 is in seconds (s), meaning that CF will not re-fetch resources from the origin for 43200 seconds.

img

Cloudflare's SSL settings need to use full, as Backblaze only provides HTTPS access.

img

Of course, if you think the file link is too long and not aesthetically pleasing, you can also use Cloudflare's built-in page rule 301 to shorten the link, for example, accessing https://img.coca.cc/img.jpg directly to https://f001.backblazeb2.com/file/imgsgj/img.jpg.
Example:
Matching rule https://img.coca.cc/*
Redirect URL to https://f001.backblazeb2.com/file/imgsgj/$1

img

Add a page cache rule.

img

Finally, please note: The traffic from Backblaze B2 to Cloudflare is free, meaning that even if CF fetches from B2, there is no $0.01/GB outbound traffic fee, equivalent to free unlimited traffic. However, this does not prevent bad actors from directly using the B2 domain (instead of the CF domain) to consume traffic.

Last modified: March 1, 2021

Article 2: Backblaze B2 + Cloudflare CDN + PicGo#

Preface#

Recently, due to the need to expand server functionality and the inconvenience of pushing logs with Alibaba Cloud, I decided to purchase a new server and deploy related websites and services. I decided to take this opportunity to standardize image storage. Since the website is not filed, services provided by domestic vendors like Alibaba Cloud OSS + Alibaba Cloud CDN, Youpai Cloud, Qiniu Cloud, etc., cannot be enjoyed, so I can only turn my attention abroad.

Originally, using GitHub + JSDelivr was a good free image hosting solution, and there are many tutorials online. However, the new usage policy released by JSDelivr on August 9, 2020, raised discussions, particularly the fourth point:

4. Prohibited Use

The following behavior is prohibited:

 1. Hosting or accessing content that:
     - contains malware or harmful code in any form,
     - violates proprietary rights of others,
     - is sexually explicit,
     - is potentially illegal in the EU or the USA.

 2. Abusing the service and its resources, or using jsDelivr as a general-purpose
    file or media hosting service. This includes, for example:
     - running an image hosting website and using jsDelivr as a storage for all
       uploaded images,
     - hosting videos, file backups, or other files in large quantities.

    We recognize that there are legitimate projects that consist of a large number
    of files, and these are not considered abuse. For example: icons packs, apps,
    or games with a large number of assets.Copy

The phrase running an image hosting website and using jsDelivr as a storage for all uploaded images is somewhat ambiguous because it is unclear whether the number of images uploaded to a personal blog and their access volume would be deemed as an image storage site, leading to a ban or deletion of data, so I could only seek other solutions.

At this time, I discovered that Cloudflare Bandwidth Alliance includes Backblaze, and all outbound traffic from Backblaze to Cloudflare is completely free. Additionally, Backblaze offers personal users 10G of free storage, 1G of daily download, and unlimited upload. We can use both to achieve 10G free quota and unlimited upload and download.

3961390e212619b300d72abb1c6d7467-NyquapNRbB

After exceeding the free storage quota, Backblaze's prices are also relatively low compared to other mainstream OSS service providers abroad, with the only downside being that it currently only supports credit card payments.

92f663d63338db8becab34b40760b0c3-i5xUgrOY5D

Prerequisites#

  1. A domain name~ preferably a primary domain name, such as xxx.xxx. You can purchase from any domain provider, and NameSilo is recommended, as it supports Alipay payments, offers free Whois privacy protection, and is inexpensive. A .com domain name costs only $9.95 per year, and if you don't mind, you can buy other suffixes for just $2 a year. There are also some free domain names available for application, the most famous being Freenom's domains, but I heard that this year Cloudflare's free service no longer supports free domains, so you can only try to see if it works.

Backblaze OSS Part#

  1. Go to this link to create a Backblaze account and complete the relevant information. You may not see the Sign Up option if you enter directly from the homepage.
  2. Go to this link to access the console and create a Buckets storage bucket.
  3. Name the bucket something easy to recognize and hard to guess. You don't need to worry about the length of the generated image link, as it can be redirected later for easier recognition. Just ensure that Files in Bucket are is set to Public.
    Note: When naming the bucket, try to choose a name that is not easily guessed by others. You can add some random characters, such as c42rx71-example-winer-website, because when the bucket is set to Public, anyone only needs to know your OSS service provider + bucket name + one of the file names to piece together the real link to the file, thus bypassing the CDN and consuming your free traffic. If you have a payment method bound, you will incur charges.

80ddd93eafadbb256b763fa1ae4e0757-image-20210914161619618

  1. Click Upload/Download to upload an image or temporary file to obtain the server address of the bucket.
  2. From the left menu, go to Browse Files, enter the bucket you just created, and find the uploaded file. Click to view details and remember the link content in the Friendly URL line.

a875ac31d761acd0f2d4e23a2c4d500d-image-20210914161911330

  1. From the Friendly URL, you can see that our bucket is located at https://f002.backblazeb2.com/, and /file/example-winer-website is the relative path where the files are stored. Remember these two values and log in to Cloudflare to set them up.

Cloudflare Part#

  1. Open the Cloudflare official website and register.
  2. Follow the prompts to change the DNS resolution server at your domain registrar to the server address provided by Cloudflare. Different users have different addresses, and for personal use, you can choose the free plan.

c4693f6f0580b728b9315b2754d10d7d-image-20210911163637613

  1. Add a CNAME record to the storage bucket server you just obtained. You can choose any secondary domain name, but generally, the following options are used:

    oss
    img
    images
    assets
    static (not only stores images but also stores css and js files used when loading the website)
    

669afcbfe869f4fb67cee2a9132a87bc-image-20210914162453761

  1. Once the DNS resolution is complete, we have completed the setup to access Backblaze through Cloudflare. You can use the browser's developer tools to access the corresponding URL under the secondary domain to check if the CDN cache is hit. The cf-cache-status line HIT indicates a cache hit, while MISS indicates no cache hit, which means it has to fetch from Backblaze, consuming download traffic.

9dab9b8dededc833583c61fb9615aaa5-image-20210911165431297

  1. To avoid cache misses or excessive fetching from the origin leading to slow loading speeds, we need to return to Backblaze to set the bucket information, adding {"cache-control":"max-age=86400"}, meaning that within 86400 seconds, Cloudflare will not re-fetch information from the origin.
    Note: When fetching from the origin, the CDN node fetches data from the origin and then delivers it to the user. It does not directly pass the origin address to the user, so there is no need to worry about excessive fetching consuming your free traffic quota. The max-age does not need to be too long; if it is too long and the source file changes without the site actively pushing to the CDN node, users may not receive the latest version in a timely manner. (While writing this, I found that BackBlaze also has a free request count display, with each Class B and Class C request limited to 2500 times per day. It is currently unclear whether Cloudflare access counts against the request count, so I do not recommend setting it too short.)

c5aa7f1948a64ce3cd9391ad316ac951-image-20210914162745460

  1. Optional not recommended redirect method: Using Cloudflare's built-in page rule 301 to redirect file links to shorten the link address. You can create it according to the options in the image below, *to match all image paths and names, where $1 represents the content of the wildcard part to complete the URL.
    Note:

1. When matching the URL, the /img in the address is necessary; otherwise, the forwarded URL will match the forwarding rule again, causing excessive redirection or other errors. However, /img can be replaced with any string that is not /file/example-winer-website (your bucket name).
2. After testing, it was found that using this method for redirection can capture the redirected link in the developer tools Network page, and directly opening the image in a new tab can also retrieve it, thus revealing your bucket name and real file name, allowing malicious traffic attacks to be constructed with the server provider. Therefore, it is recommended to use the second creation method in step 8.

da954ad89387900fa24ea3a96a9255da-image-20210914163315660

  1. Optional: When looking for information, some places suggest adding a page cache rule after performing URL redirection to prevent cache invalidation. You can configure it according to the image below. However, in my tests, I found that even without adding this content, it can still hit Cloudflare's CDN cache, so it has not been added for now. You can also increase the security level option here and set it to essentially off to ensure that when the main site is attacked and verification is added at the CDN stage, the images in the locally written Markdown document can still load normally.
    The possible reason for this step is that when the cache level is set to no query string, it only uses cached content when the image is not followed by parameters. Some OSS providers offer the ability to append parameters for image processing (such as compression, cropping, scaling) before sending. If the main site cache level is set to standard, modifying some content here can enable those features. However, Backblaze's free service does not include these, so it is not needed for now. If issues arise in the future or the real reason is found, I will provide additional explanations.

5e57fe3d244d67a57ba0e6f1b0b9256b-image-20210914164221363

  1. Optional recommended redirect method: Since neither link redirection nor page rule redirection conceals the bucket name, we choose to solve this through rewriting. This method can rewrite the content without changing the link. This time we will use the transformation rules in the rules to perform page rewriting. Match the hostname to determine the request source that needs to be rewritten, and match the complete URI again to ensure it does not contain already rewritten links to avoid repeated rewriting errors. The meaning of the concat in the rewrite path is to concatenate two strings, adding the bucket path before the requested file path, while the hostname example.winer.website does not need to be written when using the concat function, as it will automatically be added as the hostname of the incoming request. (If you must add the hostname in the concat function, do not start with http:// or https://, but replace it with /, for example, concat("/example.winer.website/file/example-winer-website", http.request.uri.path))
    Note: If the remaining page rules are insufficient or you do not want to use this method, you can refer to this article to create a private bucket and use Cloudflare Workers for access. Since the Workers access method has existed for a long time and many other bloggers have tutorials, I will not write about it here. However, Workers also have free limits, so please be cautious.

de64d3a53e203c221c0969806ad58aa6-image-20210914164843797

Using PicGo for Better Image Hosting#

First, congratulations! You have completed the basic setup of the image hosting! Below are the expansion parts.

  1. After the image hosting is set up, if no other operations are performed, every time you upload an image, you need to open Backblaze's website and enter the bucket settings page after completing identity verification, which is neither convenient nor user-friendly. Therefore, I introduce a cross-platform image upload tool that supports both command line and graphical interface—PicGo.
    Its main function is to upload images to your specified location and obtain image links. When paired with the Typora editor, it can automatically upload images and write the links into the Markdown document when pasting images, avoiding image loss caused by moving documents after images are placed on other websites or locally.
  2. Go to the project homepage, download and install PicGo.
  3. Select the plugin settings, search for s3, or go to the plugin project homepage to download and install the plugin.
  4. After installation, the image hosting settings on the left will show "Amazon S3", which can be used for all cloud storage compatible with the S3 API.
  5. Open the Backblaze official AppKeys settings page, add a new Key. It is recommended not to use the Master Application Key, and set and apply the Key in the following format.
    Note: The maximum validity period of the Key is 1000 days, which is 86400000 seconds. Please reapply after expiration.

209573ceb1bfacd10cfc6d61ae7b43fc-image-20210911203404570

  1. Record the obtained Key ID, applicationKey, bucket name, and bucket's Endpoint address and fill them in the corresponding positions. Also, fill in the URL of the 301 redirect or page rewrite into the custom domain. The other settings do not need to be changed. After saving, you can upload images, and you can also modify the naming method of the files or set it through the configuration file. For details, see this page.

Precautions#

So far, all work has been completed, but there is one thing to note:

  1. When uploading files using this method, if the file path contains spaces, Backblaze will replace the spaces with plus signs, so the links generated by PicGo after uploading files with spaces in the path will be incorrect.
  2. For the image hosting built using this method or the tutorial written, be sure not to disclose the bucket name to avoid traffic abuse, and it is recommended to add hash values to the naming rules of all uploaded files to make them unguessable, preventing access to other files when the bucket name is exposed.

At the same time, the bucket established in this article has not been used, and after writing the tutorial, it was immediately deleted, and a bucket name that is extremely difficult to guess was created, changing the relevant addresses. There is no need to attempt to access using the information in the text or images.

Last modified: August 28, 2022, 10:37 AM

Comments

Just off the plane Daoist May 24th, 2022 at 11:48 am

Thanks to the big guy for the article. Now Cloudflare provides transform rules, so you can rewrite paths without using workers.

sakuya April 9th, 2022 at 02:44 pm

Friendly reminder:
In step 8, the concat "/example.winer.website/file/example−winer−website",http.requrst.uri.paht) seems to have an extra parenthesis, and the request and path are written incorrectly.

winer August 28th, 2022 at 10:36 am

@sakuya

Thanks for the reminder. The extra parenthesis is actually due to a Chinese parenthesis outside; just look at the one with the code tag inside.
However, the request and path were indeed typed incorrectly, hhhhh. The image is still correct; I will fix that.

Article 3: Backblaze B2 + Cloudflare#

B2 is an OSS storage service provided by Backblaze, offering 10G of free capacity and 1G of free traffic per day. Moreover, it is a member of the Cloudflare Bandwidth Alliance, making the transfer of content from Backblaze through Cloudflare completely free. Even when exceeding the free quota, the price of $0.005 GB/Mo for capacity and $0.01 GB/Mo for traffic is still very cheap compared to other storage services.

Many other articles also introduce how to use B2 and CF as free image hosting, but now CF has a new transform rules feature, which provides more free entries and is easier to use compared to the Workers and page rules used in other articles.

Of course, it can also be used as a cloud disk, but 10G is not enough.

Requirements#

First, you need to register for Backblaze B2 and Cloudflare accounts, and then you need a domain name. Cloudflare now provides domain purchases, or you can buy from namesilo or other places and then resolve it with Cloudflare. After registering with Backblaze, you need to verify your email; otherwise, you cannot create a public storage bucket.

Creating a Storage Bucket#

Enter the Backblaze user management interface, select the bucket option under B2 cloud storage, and then click to create a storage bucket. You need to enter the name of the bucket during creation; this name will be included in the URL, so it cannot have the same name as someone else's bucket or your other buckets. When creating, select the public option to set it to public; other options do not need to be modified.

After creation, you can upload an image inside, click on the uploaded image, and the friendly URL will be the address to use. Next, record the hostname of the friendly URL, such as f00x.backblaze2.com.

The newly created storage bucket will not be cached by Cloudflare by default, and it will have to fetch from the origin each time. For image hosting, caching is more convenient. Click on the bucket option, return to the place where you created the bucket, and you can see the created bucket. Click on the bucket settings and change the bucket information to {"cache-control":"max-age=43200"}, where the number after max-age= is the cache duration for Cloudflare, in seconds.

Configuring Cloudflare#

Create a CNAME record in Cloudflare for the subdomain you want to use for image hosting, pointing it to the hostname you just recorded, f00x.backblaze2.com. Since you want to use Cloudflare's CDN for caching, set the proxy status to proxied.

After creating the CNAME record, replace the hostname of the friendly URL with the domain you just created to access the files. If Cloudflare prompts error 522 during testing, it is because Backblaze only allows HTTPS access. You need to change the SSL/TLS setting in Cloudflare to Full (strict). Additionally, if you are a Cloudflare Pro paid user, you also need to turn off Automatic Signed Exchanges (SXG), which is in the Speed-Optimization options.

Directly using the CNAME domain to modify the URL will expose the bucket name and make the URL very verbose. At this point, you need to use Cloudflare's transform rules to shorten the URL.

Select Rules - Transform Rules, then click Create Transform Rule - Rewrite URL to create a rewrite URL rule. First, give the rule a name. If you are using a separate subdomain as a dedicated image hosting domain, you can directly select the hostname in the field, choose equals in the operator, and then enter the subdomain in the value.

If the image hosting is not using a dedicated subdomain, you need to click Write Expression and refer to the Cloudflare documentation to write how to match the URL address of the image hosting.

After configuring the matching expression, in the rewrite - path below, select Rewrite to, and in the options, choose Dynamic. If you only want to remove the bucket name /file/ from the URL, fill in concat("/file/bucket name", http.request.uri.path) at the end. If you have other requirements, you can also refer to the Cloudflare documentation above to modify it. After setting, click Deploy to enable the rule. Now enter https://img.example.com/file path in bucket or the custom address to load the images in the bucket.

The following is an image uploaded in this way. Additionally, if someone recognizes the author, please reply in the comments.

Image omitted...

References#

Free Image Hosting With Cloudflare Transform Rules and Backblaze B2

Backblaze B2 + Cloudflare CDN customizable domain 10G high-speed free external link image hosting

Backblaze + CloudFlare to achieve zero-cost simple image hosting

Loading...
Ownership of this post data is guaranteed by blockchain and smart contracts to the creator alone.