Recently I updated a Laravel 5.7 project to use the newest version of Jeffery Way's compilation platform Laravel Mix. This update includes Webkit 4 support, faster installations and compilation processes, and a bunch of other goodies. Unfortunately this update introduced a wrench into an automated deployment process that I had in place.
My environment
Hosting
I use fortrabbit for a lot of my production applications because of their managed Laravel hosting that is relatively inexpensive compared to other offerings, and yet it is extremely reliable. For asset storage, fortrabbit employs a custom S3 bucket implementation. When I say custom, all of the normal S3 API protocols apply to this system however the standard Amazon Web Services authentication will not work because, well, it's not Amazon; duh.
While it is a great implementation, it is worth keeping in mind that a lot of the standard out of the box plugins for S3 uploads all assume you are uploading to AWS, which is not compatible with fortrabbit's S3 implementation.
Automation
To automate my deployment processes, I use this great platform called Buddy. Buddy is like that of Jenkins, Bamboo, and any of these other automation platforms; it allows you to perform dev ops functions with simply a click-of-a-button or a repository action (push, merge, etc). The exceptions are it offers an interface that much more user friendly then that most other options, and it provides pre-configured plugins that allow you to jump start your deployment processes quicker than that of Jenkins of Bamboo.
What's more, Buddy is different because Buddy uses Docker containers to run anything from Python, NodeJS, RSYNC or even Haskell on your repository's code. You can even attach MariaDB, MongoDB, or MySQL databases to these Docker containers to run unit testing.
My issue
I had been using this Webpack plug-in to automatically upload all of my compiled Javascript and CSS upon the completion of the development/production deployment. When upgrading to Mix 4, the updated S3 Webpack plugin uploads all of the resulting files in a multi-part upload, which fortrabbit does not support. So as you would expect my automated deployments began producing errors.
Solution
Below is a screenshot of my Buddy pipeline for my production deployment:
Conventional wisdom would suggest that I just utilize Buddy's default S3 upload plugin. While you would be right in most cases, but as I said earlier, most S3 upload solutions will not work with fortrabbit because you need to be able to provide custom url endpoints and custom authentication protocols.
Configuration
In the screenshot above, you can see that instead of an AWS-branded solution, I have decided to use stock Linux Python v3.5 Docker container with the following settings:
Using PIP, I installed the official AWS CLI and then configured it using the S3 credentials that were supplied to me by fortrabbit; hence the various aws configure set commands. Here are a few other things to pay attention to:
- The fourth configuration setting (
aws configure set output text) simply tells AWS that I want all of the results to be outputted to text; which is perfect for simple automation - The fifth and sixth configuration settings tell the CLI that I want to use the new V4 authentication protocol. This is important. If you don't use these settings, you will get an error saying that the signature is invalid.
Run the configuration
After the container has been configured, now I can run the command that will upload the necessary assets to S3:
aws s3 cp public/build/ s3://gm-production/build/ --recursive --exclude ".DS_Store" --endpoint-url=https://objects.us1.frbit.comSo lets break this down:
- Skipping the
aws s3portion, as that should be pretty self-explanatory, the next portion should look pretty familiar to those who use BASH commands:cp public/build/ s3://gm-production/build/. Essentially you are copying a directory to a url using the S3 protocol. --recursivetells the CLI that you want to copy the directory in its entirety; including any additional sub directories.- You can tell AWS if there are any files/directories that you don't want to copy. You can do this using a regular expression. In my case:
--exclude ".DS_Store"I am telling the CLI that I don't want to copy those pesky DS_Store files that the MacOS generates. - Finally, and possibly the most important, is the endpoint url for the bucket:
--endpoint-url=https://objects.us1.frbit.com. With custom S3 buckets this url has to be added to the request, or the CLI will think that you are going to be uploading to an AWS based bucket.
Uploading your files
Assuming that you have everything hooked up correctly, your upload process that is returned by Buddy should look something like this:
aws s3 cp public/build/ s3://gm-production/build/ --recursive --exclude .DS_Store --endpoint-url=https://objects.us1.frbit.com
Completed 256.0 KiB/6.2 MiB (1.3 MiB/s) with 3 file(s) remaining
Completed 512.0 KiB/6.2 MiB (2.2 MiB/s) with 3 file(s) remaining
Completed 768.0 KiB/6.2 MiB (3.3 MiB/s) with 3 file(s) remaining
Completed 1.0 MiB/6.2 MiB (4.4 MiB/s) with 3 file(s) remaining
Completed 1.2 MiB/6.2 MiB (5.2 MiB/s) with 3 file(s) remaining
Completed 1.5 MiB/6.2 MiB (5.9 MiB/s) with 3 file(s) remaining
Completed 1.8 MiB/6.2 MiB (6.8 MiB/s) with 3 file(s) remaining
Completed 2.0 MiB/6.2 MiB (7.8 MiB/s) with 3 file(s) remaining
Completed 2.0 MiB/6.2 MiB (7.5 MiB/s) with 3 file(s) remaining
upload: public/build/css/vendor.css to s3://gm-testing/build/css/vendor.css
Completed 2.0 MiB/6.2 MiB (7.5 MiB/s) with 2 file(s) remaining
Completed 2.3 MiB/6.2 MiB (7.8 MiB/s) with 2 file(s) remaining
Completed 2.5 MiB/6.2 MiB (8.2 MiB/s) with 2 file(s) remaining
Completed 2.8 MiB/6.2 MiB (8.6 MiB/s) with 2 file(s) remaining
Completed 3.0 MiB/6.2 MiB (8.3 MiB/s) with 2 file(s) remaining
Completed 3.3 MiB/6.2 MiB (8.9 MiB/s) with 2 file(s) remaining
Completed 3.5 MiB/6.2 MiB (9.3 MiB/s) with 2 file(s) remaining
upload: public/build/css/main.css to s3://gm-testing/build/css/main.css
Completed 3.5 MiB/6.2 MiB (9.3 MiB/s) with 1 file(s) remaining
Completed 3.7 MiB/6.2 MiB (9.7 MiB/s) with 1 file(s) remaining
Completed 4.0 MiB/6.2 MiB (9.3 MiB/s) with 1 file(s) remaining
Completed 4.2 MiB/6.2 MiB (9.8 MiB/s) with 1 file(s) remaining
Completed 4.5 MiB/6.2 MiB (9.8 MiB/s) with 1 file(s) remaining
Completed 4.7 MiB/6.2 MiB (10.0 MiB/s) with 1 file(s) remaining
Completed 5.0 MiB/6.2 MiB (9.7 MiB/s) with 1 file(s) remaining
Completed 5.2 MiB/6.2 MiB (9.6 MiB/s) with 1 file(s) remaining
Completed 5.5 MiB/6.2 MiB (9.2 MiB/s) with 1 file(s) remaining
Completed 5.7 MiB/6.2 MiB (8.8 MiB/s) with 1 file(s) remaining
Completed 6.0 MiB/6.2 MiB (7.9 MiB/s) with 1 file(s) remaining
Completed 6.2 MiB/6.2 MiB (6.7 MiB/s) with 1 file(s) remaining
Completed 6.2 MiB/6.2 MiB (2.3 MiB/s) with 1 file(s) remaining
upload: public/build/js/app.js to s3://gm-testing/build/js/app.js
Build finished successfully!.Get Buddy
Has this article interested you in Buddy? Follow this link to get signed up for free!