Table of Contents
Overview
It’s 2023 and nobody is storing files on their server anymore. There are many choices available. For me, I choose AWS S3 since I have it and it’s one of the cheapest available (as far as I know).
In this post, we are going to configure a bucket and then write an API to upload files to that bucket.
Let’s get started.
Create an AWS S3 Bucket
It’s quite straightforward to create a bucket to store files with AWS S3:
As you can see, I left everything to default values.
Hit Create bucket to create that bucket.
After creating the bucket, click on it and go to properties to get the bucket’s ARN
Configure IAM for AWS S3 bucket
Go to IAM->User and click on Create User
In the next screen, I select attach policy directly and create a new policy:
On the create policy window, I put the following details:
You can see that in the Resource section, that’s the ARN of the bucket I created before.
Hit next and now I can create the policy:
Back to the create IAM account, you need to click on the refresh button and search for the policy name:
Select the policy and finish the account creation:
Now click on the newly created user and go to Permission tab, scroll down to create an access key pair:
On the next screen, select this:
Click on create and you will see the following screen:
You can reveal the secret or download the CSV file. I’ll download the CSV file.
That’s it! Now we have the bucket and the secret key pair to upload files to S3.
Create API to upload to S3
Back to our Spring Boot application, in the environment file, enter the S3 details:
S3_BUCKET=datmt-wallet-bucket S3_ENDPOINT=https://s3.us-east-1.amazonaws.com S3_REGION=us-east-1 S3_ACCESS_KEY=AKIA4......NLU3CC5O S3_SECRET_ACCESS_KEY=w6rJJ..............FrZrmNluVa6
In pom.xml, we need to add the necessary dependency:
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-s3</artifactId>
<version>1.12.351</version>
</dependency>Next, let’s create the helper class to upload files to S3
public class AwsS3File {
private static final String BUCKET = System.getenv("S3_BUCKET");
private static final String REGION = System.getenv("S3_REGION");
private static final String ACCESS_KEY = System.getenv("S3_ACCESS_KEY");
private static final String ENDPOINT = System.getenv("S3_ENDPOINT");
private static final String SECRET_ACCESS_KEY = System.getenv("S3_SECRET_ACCESS_KEY");
private final AmazonS3 s3Client;
public AwsS3File() {
AWSCredentialsProvider cred = new AWSStaticCredentialsProvider(
new BasicAWSCredentials(ACCESS_KEY, SECRET_ACCESS_KEY)
);
s3Client =
AmazonS3ClientBuilder
.standard()
.withEndpointConfiguration(
new AwsClientBuilder.EndpointConfiguration(ENDPOINT, REGION)
)
.withCredentials(cred)
.build();
}
public PutObjectResult uploadObjectPrivate(File file, String spaceFilePath) {
PutObjectRequest putObjectRequest = new PutObjectRequest(
BUCKET,
spaceFilePath,
file
);
return s3Client.putObject(putObjectRequest);
}
public void deleteObject(String objectKey) {
var deleteRequest = new DeleteObjectRequest(BUCKET, objectKey);
s3Client.deleteObject(deleteRequest);
}
public URL generateDownloadURL(String objectKey, int expirationInMinutes) {
return s3Client.generatePresignedUrl(
BUCKET,
objectKey,
new Date(System.currentTimeMillis() + expirationInMinutes * 60000L),
HttpMethod.GET
);
}
public URL generateDownloadURL(String objectKey) {
return generateDownloadURL(objectKey, 180);
}
public byte[] getObjectByte(String objectKey) throws IOException {
return s3Client.getObject(BUCKET, objectKey).getObjectContent().readAllBytes();
}
}
We also need to create a document to store the uploaded file. We want to hide the s3 path and manage access to the uploaded file. In case of file deletion, we want to make sure that only the owner can do that.
@Document(collection = "files")
@Data
public class FileUpload implements Serializable {
@MongoId
private String id;
private String s3Path;
private String ownerId;
}
@Repository
public interface FileUploadRepository extends MongoRepository<FileUpload, String> {
}
Let’s also create a service to handle the file upload instead of putting all in the controller:
@Service
@RequiredArgsConstructor
public class FileUploadService {
private final FileUploadRepository fileUploadRepository;
private final CurrentUserService currentUserService;
public File multipartToFile(MultipartFile multipartFile) throws IOException {
File file = File.createTempFile("temp", null);
multipartFile.transferTo(file);
return file;
}
public FileUpload upload(MultipartFile multipartFile) throws IOException {
var file = multipartToFile(multipartFile);
var storageFilePath = UUID.randomUUID() + multipartFile.getOriginalFilename();
var awsS3File = new AwsS3File();
var result = awsS3File.uploadObjectPrivate(file, storageFilePath);
if (result.getContentMd5() == null)
throw new IOException("Upload failed");
var fileUpload = new FileUpload();
fileUpload.setOwnerId(currentUserService.getCurrentUserId());
fileUpload.setS3Path(storageFilePath);
return fileUploadRepository.save(fileUpload);
}
}
Now let’s create the endpoint to upload files to S3:
@RestController
@RequestMapping("/api/files")
@RequiredArgsConstructor
public class FileUploadController {
private final FileUploadService fileUploadService;
@PostMapping
public FileUpload upload(
@RequestParam("file") MultipartFile file
) throws IOException {
return fileUploadService.upload(file);
}
}It’s time to test our flow:
As you can see, the upload was successful. We have all we need to develop the UI.
Conclusion
In this post, we’ve created the API to upload images to S3. In the next post, we are going to write the UI to complete the application.
I build softwares that solve problems. I also love writing/documenting things I learn/want to learn.