Let's create an environment that scans a file via an S3 event by utilizing ClamAV binaries on a Lambda layer. You can retrieve the full source code at this GitHub repository.

Serverless config

So, we'll need a few things: an S3 bucket, a function, and a lambda layer. I've also included logging groups, and permissions in the serverless.yml file:

service: clambda-av

provider:
  name: aws
  runtime: nodejs14.x
  ecr:
    images:
      clambdaAv:
        path: ./
  iamRoleStatements:
    - Effect: Allow
      Action:
        - s3:GetObject
        - s3:PutObjectTagging
      Resource: "arn:aws:s3:::clambda-av-files/*"

functions:
  virusScan:
    image:
      name: clambdaAv
    memorySize: 2048
    events:
      - s3: 
          bucket: clambda-av-files
          event: s3:ObjectCreated:*
    timeout: 120

package:
  exclude:
    - node_modules/**
    - coverage/**

Dockerfile

Before we deploy this, we need to build our ClamAV binaries and put them, along with our handler, into a container. We can do this with a multi-stage Docker image. It will first build the ClamAV binaries and package them with the virus definitions, then build the lambda container and pull in the packaged ClamAV files:

FROM amazonlinux:2 AS layer-image

WORKDIR /home/build

RUN set -e

RUN echo "Prepping ClamAV"

RUN rm -rf bin
RUN rm -rf lib

RUN yum update -y
RUN amazon-linux-extras install epel -y
RUN yum install -y cpio yum-utils tar.x86_64 gzip zip

RUN yumdownloader -x \*i686 --archlist=x86_64 clamav
RUN rpm2cpio clamav-0*.rpm | cpio -vimd

RUN yumdownloader -x \*i686 --archlist=x86_64 clamav-lib
RUN rpm2cpio clamav-lib*.rpm | cpio -vimd

RUN yumdownloader -x \*i686 --archlist=x86_64 clamav-update
RUN rpm2cpio clamav-update*.rpm | cpio -vimd

RUN yumdownloader -x \*i686 --archlist=x86_64 json-c
RUN rpm2cpio json-c*.rpm | cpio -vimd

RUN yumdownloader -x \*i686 --archlist=x86_64 pcre2
RUN rpm2cpio pcre*.rpm | cpio -vimd

RUN yumdownloader -x \*i686 --archlist=x86_64 libtool-ltdl
RUN rpm2cpio libtool-ltdl*.rpm | cpio -vimd

RUN yumdownloader -x \*i686 --archlist=x86_64 libxml2
RUN rpm2cpio libxml2*.rpm | cpio -vimd

RUN yumdownloader -x \*i686 --archlist=x86_64 bzip2-libs
RUN rpm2cpio bzip2-libs*.rpm | cpio -vimd

RUN yumdownloader -x \*i686 --archlist=x86_64 xz-libs
RUN rpm2cpio xz-libs*.rpm | cpio -vimd

RUN yumdownloader -x \*i686 --archlist=x86_64 libprelude
RUN rpm2cpio libprelude*.rpm | cpio -vimd

RUN yumdownloader -x \*i686 --archlist=x86_64 gnutls
RUN rpm2cpio gnutls*.rpm | cpio -vimd

RUN yumdownloader -x \*i686 --archlist=x86_64 nettle
RUN rpm2cpio nettle*.rpm | cpio -vimd

RUN mkdir -p bin
RUN mkdir -p lib
RUN mkdir -p var/lib/clamav
RUN chmod -R 777 var/lib/clamav

COPY ./freshclam.conf .

RUN cp usr/bin/clamscan usr/bin/freshclam bin/.
RUN cp usr/lib64/* lib/.
RUN cp freshclam.conf bin/freshclam.conf

RUN yum install shadow-utils.x86_64 -y

RUN groupadd clamav
RUN useradd -g clamav -s /bin/false -c "Clam Antivirus" clamav
RUN useradd -g clamav -s /bin/false -c "Clam Antivirus" clamupdate

RUN LD_LIBRARY_PATH=./lib ./bin/freshclam --config-file=bin/freshclam.conf

FROM public.ecr.aws/lambda/nodejs:14

COPY --from=layer-image /home/build ./

COPY handler.js ./

CMD ["handler.virusScan"]

You'll note that the COPY ./freshclam.conf . line implies that there's another file that we need and you'd be correct:

DatabaseMirror database.clamav.net
CompressLocalDatabase yes
ScriptedUpdates no
DatabaseDirectory /opt/var/lib/clamav

Scanning Handler

Furthermore, we also need our handler to do the scanning and tagging of the file. The tagging is more of a placeholder for whatever flow you'd want to implement. You can create a separate quarantine bucket or a separate clean bucket, or both - whichever you prefer.

Our handler:

const { execSync } = require("child_process");
const { writeFileSync, unlinkSync } = require("fs");
const AWS = require("aws-sdk");

const s3 = new AWS.S3();

module.exports.virusScan = async (event, context) => {
  if (!event.Records) {
    console.log("Not an S3 event invocation!");
    return;
  }

  for (const record of event.Records) {
    if (!record.s3) {
      console.log("Not an S3 Record!");
      continue;
    }

    // get the file
    const s3Object = await s3
      .getObject({
        Bucket: record.s3.bucket.name,
        Key: record.s3.object.key
      })
      .promise();

    // write file to disk
    writeFileSync(`/tmp/${record.s3.object.key}`, s3Object.Body);

    try { 
      // scan it
      execSync(`./bin/clamscan --database=./var/lib/clamav /tmp/${record.s3.object.key}`);

      await s3
        .putObjectTagging({
          Bucket: record.s3.bucket.name,
          Key: record.s3.object.key,
          Tagging: {
            TagSet: [
              {
                Key: 'av-status',
                Value: 'clean'
              }
            ]
          }
        })
        .promise();
    } catch(err) {
      if (err.status === 1) {
        // tag as dirty, OR you can delete it
        await s3
          .putObjectTagging({
            Bucket: record.s3.bucket.name,
            Key: record.s3.object.key,
            Tagging: {
              TagSet: [
                {
                  Key: 'av-status',
                  Value: 'dirty'
                }
              ]
            }
          })
          .promise();
      }
    }

    // delete the temp file
    unlinkSync(`/tmp/${record.s3.object.key}`);
  }
};

Here's the algorithm:

  • If a file is clean, then it is tagged with a key/value pair of av-status = 'clean'
  • If a file is NOT clean (virus), then it is tagged with a key/value pair of av-status = 'dirty'

Since this is for demonstration purposes, you can obviously customize this flow however you'd like. 😀

Deploying

Now that we have all of our files, we can deploy our application via sls deploy:

Serverless: Packaging service...
#1 [internal] load build definition from Dockerfile
#1 sha256:f7cf3ae7c8fedea71f1d39e782987c67445d5da3084891172f6594fd2a4d6e6a
#1 transferring dockerfile: 37B 0.0s done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 sha256:9ea183ef2d94cbb0a824801e98312bcf6009b4baba73985e735ca40b3921cb2e
#2 transferring context: 2B done
#2 DONE 0.0s

#4 [internal] load metadata for docker.io/library/amazonlinux:2
#4 sha256:fc74cb88fb41757238b7c8f807a8a54ab36c03b02384db53ca30ebf4e8a96d10
#4 DONE 0.9s

#3 [internal] load metadata for public.ecr.aws/lambda/nodejs:14
#3 sha256:1e7d9f35caa3cda46944c97429a62606caffcc393fbcf77177c9995bf747587e
#3 DONE 0.9s

#6 [layer-image  1/46] FROM docker.io/library/amazonlinux:2@sha256:93e55c026fe5dba304ff3f24690b07cb39f610b160844a575bb8f3fec41b3a5e
#6 sha256:0ee538d609bad566cb8282b3544a025c0925bcfcb50a540c86d1307aae7ead58
#6 DONE 0.0s

#5 [stage-1 1/3] FROM public.ecr.aws/lambda/nodejs:14@sha256:d254aa26128c37173fc3f966539e03fe51ce82b251c7b937621bfece9e16e535
#5 sha256:7dae886c22227978cedb9fc7704338c8bc83684c46a4c72ac44f78057f530b20
#5 DONE 0.0s

#43 [internal] load build context
#43 sha256:86a21d9c36d71661a7c7ae6251b0070375769ee446c840e0506882275ed2615f
#43 transferring context: 65B done
#43 DONE 0.0s

#31 [layer-image 26/46] RUN yumdownloader -x *i686 --archlist=x86_64 xz-libs
#31 sha256:3f665ff1aec15fd2863120dc6ce8dc4699b846ac5b51e40f61e9066e103dc300
#31 CACHED

#35 [layer-image 30/46] RUN yumdownloader -x *i686 --archlist=x86_64 gnutls
#35 sha256:74889bfca401670f98add2eec605ab98485933830cbb78bae9f6480b9f7ab62c
#35 CACHED

#30 [layer-image 25/46] RUN rpm2cpio bzip2-libs*.rpm | cpio -vimd
#30 sha256:a1e48c656a1742202a06f9ddef4161d1c64e247a1c4440c37cc18d3cfda241f3
#30 CACHED

#23 [layer-image 18/46] RUN yumdownloader -x *i686 --archlist=x86_64 pcre2
#23 sha256:77e829f42bbae638469f3aa684d479496f8d0e0b7483d252e165db9b17d2dac7
#23 CACHED

#33 [layer-image 28/46] RUN yumdownloader -x *i686 --archlist=x86_64 libprelude
#33 sha256:4b8b0602af3a49dfb0f57c219394c8210965dd5d474d1ef63dc1d690b5f7479e
#33 CACHED

#22 [layer-image 17/46] RUN rpm2cpio json-c*.rpm | cpio -vimd
#22 sha256:6cfd795ecee5ea6a13c8102e4750bec0f3a19b8dceb96a2b85276378500dc510
#22 CACHED

#44 [layer-image 38/46] COPY ./freshclam.conf .
#44 sha256:3e9f51619718abc65c59a9598f0097945f3f22c269a250c3b8e14cebd69adff8
#44 CACHED

#26 [layer-image 21/46] RUN rpm2cpio libtool-ltdl*.rpm | cpio -vimd
#26 sha256:4ca322fdc49602d9a64458e5e2dc91fc25618191c2153c32da5c5589530564e4
#26 CACHED

#42 [layer-image 37/46] RUN chmod -R 777 var/lib/clamav
#42 sha256:8b90d59ac1a117f58843f29799b9814edbd785202ea0faeb1832b93229f31a8f
#42 CACHED

#45 [layer-image 39/46] RUN cp usr/bin/clamscan usr/bin/freshclam bin/.
#45 sha256:60d48c88a765a6924d8e1b4cae682544dfffda475cf919958bcff32f5138b634
#45 CACHED

#17 [layer-image 12/46] RUN yumdownloader -x *i686 --archlist=x86_64 clamav-lib
#17 sha256:ba99ffa74462e1eeee040648830000626982feb9dc39f89ea378d17ab8ffb733
#17 CACHED

#15 [layer-image 10/46] RUN yumdownloader -x *i686 --archlist=x86_64 clamav
#15 sha256:b9879128d05995548525bb63b4353629886721e033f078a49029e528c24e7c7d
#15 CACHED

#20 [layer-image 15/46] RUN rpm2cpio clamav-update*.rpm | cpio -vimd
#20 sha256:79ead81f8667e54e224cdc676a3c1b1372847a8b5bec8c4f1660df0e2437ec13
#20 CACHED

#48 [layer-image 42/46] RUN yum install shadow-utils.x86_64 -y
#48 sha256:50c00e16ca24e2923b39fe96937198dbbfd8f6e482885510e757fa7b989c2091
#48 CACHED

#39 [layer-image 34/46] RUN mkdir -p bin
#39 sha256:4b4fe13023ab760d23a9fdbf1f64d545a1dbfd36c7657018502bff0e5e7534a0
#39 CACHED

#46 [layer-image 40/46] RUN cp usr/lib64/* lib/.
#46 sha256:b3c633c8688fbe3a415ae4143911be554688f51ca7109845ddf8a960d1b8560d
#46 CACHED

#10 [layer-image  5/46] RUN rm -rf bin
#10 sha256:1ab30e2e64b59f410a34c394e4847fd6f66fa74e0d4ca05ef096ddf0c849ad4c
#10 CACHED

#51 [layer-image 45/46] RUN useradd -g clamav -s /bin/false -c "Clam Antivirus" clamupdate
#51 sha256:6d1eaffd2573e3eaa3ba74f9f5eed360da9b4762375637f5899578219dbcd740
#51 CACHED

#8 [layer-image  3/46] RUN set -e
#8 sha256:7dc57f0f956f02138ed795517511ad35a018a3ed4d9e32464bf5e7c5fc59b9d5
#8 CACHED

#13 [layer-image  8/46] RUN amazon-linux-extras install epel -y
#13 sha256:584b114c6c1152932784ffe4e16fb715627e4bccc6a1311f0f3cc0acafac1d37
#13 CACHED

#16 [layer-image 11/46] RUN rpm2cpio clamav-0*.rpm | cpio -vimd
#16 sha256:db5ced6013043eafcedaa315f21daf6434c42fc600997a0a643fd54a4ce9b82f
#16 CACHED

#7 [layer-image  2/46] WORKDIR /home/build
#7 sha256:26b9f19bcb96f2c41f071106f469e42024d6be94975fcbbc842fbd04829a6455
#7 CACHED

#19 [layer-image 14/46] RUN yumdownloader -x *i686 --archlist=x86_64 clamav-update
#19 sha256:e7fca0c79e1f2f39ed719ac21c5e1827fa28a6185f1f9813925b9bc876257975
#19 CACHED

#50 [layer-image 44/46] RUN useradd -g clamav -s /bin/false -c "Clam Antivirus" clamav
#50 sha256:ee92a39e47567e27e55be24a0b76fe8799f326ae803590f704bad3150092d37d
#50 CACHED

#47 [layer-image 41/46] RUN cp freshclam.conf bin/freshclam.conf
#47 sha256:535b454f3c9d03ab948b6744d5af9788daed9e53b62dceb348026835197f64e2
#47 CACHED

#14 [layer-image  9/46] RUN yum install -y cpio yum-utils tar.x86_64 gzip zip
#14 sha256:64b4b3193ad8e3f5425fa3b0f9c22a3c6bda2b6ec99db30fabd2b3635143d533
#14 CACHED

#37 [layer-image 32/46] RUN yumdownloader -x *i686 --archlist=x86_64 nettle
#37 sha256:68303ca07d58c666691498c2185f665f1497076f9316f3b28a9d60d2ab9ca6df
#37 CACHED

#29 [layer-image 24/46] RUN yumdownloader -x *i686 --archlist=x86_64 bzip2-libs
#29 sha256:cd7d003e83f98ec0fbb58236dd41bb31e2e13144d78edb081594fb50222ff158
#29 CACHED

#53 [stage-1 2/3] COPY --from=layer-image /home/build ./
#53 sha256:8bd3f2503c7b1c1d694cd5543b44ca22bb8bd0933ebfcfdedb0a676a64b08401
#53 CACHED

#28 [layer-image 23/46] RUN rpm2cpio libxml2*.rpm | cpio -vimd
#28 sha256:bc381c5413db49f2c0512048998f1168655af8511c98f78c56f59dd9f9604559
#28 CACHED

#52 [layer-image 46/46] RUN LD_LIBRARY_PATH=./lib ./bin/freshclam --config-file=bin/freshclam.conf
#52 sha256:d5591c20e276fbd12b5019027ea76d5c71f6390998b089e53f84c0fb4699d4cd
#52 CACHED

#25 [layer-image 20/46] RUN yumdownloader -x *i686 --archlist=x86_64 libtool-ltdl
#25 sha256:6c9ebbb8efb2ac01852c21fbff5cb1ab1789a4f9cbbf3545612b5dd8798ccd2d
#25 CACHED

#27 [layer-image 22/46] RUN yumdownloader -x *i686 --archlist=x86_64 libxml2
#27 sha256:80a8312db9cfe0fe67ba02d2832867431950b6557d6289421b7d085257ca906a
#27 CACHED

#36 [layer-image 31/46] RUN rpm2cpio gnutls*.rpm | cpio -vimd
#36 sha256:6236229072512ba7a0218cde6e4e8cfd96de21c9db77f558ad7256b270610d41
#36 CACHED

#40 [layer-image 35/46] RUN mkdir -p lib
#40 sha256:17409fa7425d5c020ccca548156f73acc43bfd0d5cea212ecc2496c102649b01
#40 CACHED

#18 [layer-image 13/46] RUN rpm2cpio clamav-lib*.rpm | cpio -vimd
#18 sha256:c0423def5fb646d7eb93a29005f7c230c90426b19e515b12132b74d9950a7b0a
#18 CACHED

#24 [layer-image 19/46] RUN rpm2cpio pcre*.rpm | cpio -vimd
#24 sha256:e306302a09b4594fab8d47bcbd6220b5dbd5a66c01f3154a10ad59a49b58072d
#24 CACHED

#32 [layer-image 27/46] RUN rpm2cpio xz-libs*.rpm | cpio -vimd
#32 sha256:19fa26ae6318b80574d2b63b71fffa000354f4bf3f90d746a2aa5ea942eafed4
#32 CACHED

#21 [layer-image 16/46] RUN yumdownloader -x *i686 --archlist=x86_64 json-c
#21 sha256:fa2a1ea71f2d3a95b6ffbd3f63a990a62836fcb5ce81a96dc37137b59dbc1cf5
#21 CACHED

#9 [layer-image  4/46] RUN echo "Prepping ClamAV"
#9 sha256:15420fa82e104178f7d3c961c97e338f87c592579b59d34ceab7e00b4633d2eb
#9 CACHED

#11 [layer-image  6/46] RUN rm -rf lib
#11 sha256:961ff93c80d43073658f8d6a884f4c27d156a1c81fa4be59ed2b8f3a75d1232d
#11 CACHED

#38 [layer-image 33/46] RUN rpm2cpio nettle*.rpm | cpio -vimd
#38 sha256:8f08172c10c21ea571f4b5f409874fdeb7b9ce50a8b0b9feea9e93974dec37d1
#38 CACHED

#41 [layer-image 36/46] RUN mkdir -p var/lib/clamav
#41 sha256:54fcb454911d11e763ba0d1db4fc91b6ff12efd6041f2d5d37b47cc935bfc6a1
#41 CACHED

#12 [layer-image  7/46] RUN yum update -y
#12 sha256:94f252f86450a307b81c0d178682036b9c40792fa353f29e7a3584210c94b375
#12 CACHED

#49 [layer-image 43/46] RUN groupadd clamav
#49 sha256:bc67447ad41a36a646004b61df20e1a50da05d201f37ec96c04b2d88215f81b0
#49 CACHED

#34 [layer-image 29/46] RUN rpm2cpio libprelude*.rpm | cpio -vimd
#34 sha256:cf604d8ff1a2c96ff77494e9b74a3235c139beb20b12685e1e4454d24ea2123f
#34 CACHED

#54 [stage-1 3/3] COPY handler.js ./
#54 sha256:d91d843e3a65e5ce02345406e5e66d5ba3f3103b034099a291a48900ceb8ba66
#54 CACHED

#55 exporting to image
#55 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
#55 exporting layers done
#55 writing image sha256:1eda3f378984fa9f4182e0ec57b70593bbf662cf1bf9e683124ea806ae6cfd1b done
#55 naming to docker.io/library/serverless-clambda-av-dev:clambdaAv done
#55 DONE 0.0s
The push refers to repository [877639635780.dkr.ecr.us-east-1.amazonaws.com/serverless-clambda-av-dev]
431b853b7050: Preparing
14ed58d332cd: Preparing
5dcc5d6d99a6: Preparing
3a28fc993254: Preparing
f6ae2f36d5d7: Preparing
49272d3ef1cf: Preparing
4fbd0611a17e: Preparing
35e2a1be67d7: Preparing
4fbd0611a17e: Waiting
35e2a1be67d7: Waiting
49272d3ef1cf: Waiting
431b853b7050: Pushed
49272d3ef1cf: Pushed
4fbd0611a17e: Pushed
f6ae2f36d5d7: Pushed
5dcc5d6d99a6: Pushed
3a28fc993254: Pushed
35e2a1be67d7: Pushed
14ed58d332cd: Pushed
clambdaAv: digest: sha256:f28c5553c848c5065bf42b1c191a59cbc77f247d54ade2dd03252827f12b66ac size: 2000
Serverless: Creating Stack...
Serverless: Checking Stack create progress...
........
Serverless: Stack create finished...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
.....................
Serverless: Stack update finished...
Service Information
service: clambda-av
stage: dev
region: us-east-1
stack: clambda-av-dev
resources: 8
api keys:
  None
endpoints:
  None
functions:
  virusScan: clambda-av-dev-virusScan
layers:
  None

Now, let's test it by uploading a clean file to S3. I happen to have a PDF laying around:

joseph@bertha > aws s3 cp ~/document.pdf s3://clambda-av-files/
upload: ../../document.pdf to s3://clambda-av-files/document.pdf

The downside is that clamscan in the Lambda layer takes ~30 seconds or so boot, load the virus definitions, and scan the file with the results. We can check the tag after some time via:

joseph@bertha > aws s3api get-object-tagging --bucket clambda-av-files --key document.pdf

{
    "TagSet": [
        {
            "Key": "av-status",
            "Value": "clean"
        }
    ]
}

We can test the virus scanner with a test virus signature found at EICAR:

X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*

After saving that text to a file called test-virus.pdf. Let's upload it and see what happens:

joseph@bertha > aws s3 cp ~/test-virus.pdf s3://clambda-av-files/
upload: ../../test-virus.pdf to s3://clambda-av-files/test-virus.pdf

After waiting another thirty seconds or so, let's check the tag on it:

joseph@bertha > aws s3api get-object-tagging --bucket clambda-av-files --key test-virus.pdf

{
    "TagSet": [
        {
            "Key": "av-status",
            "Value": "dirty"
        }
    ]
}

Thanks

Thank y'all for reading. If you have any questions, feel free to ask in the comments! I also welcome suggestions. 🙂

Logo
云原生

云原生社区为您提供最前沿的新闻资讯和知识内容

更多推荐

对全局端点的需求

在这篇文章中,我将探讨全局端点的概念,并就构建多区域应用程序所需的内容分享我的观点。 作为系列多区域道路的一部分,您可以查看其他部分: 第 1 部分- 反思在开始多区域架构之前要考虑的事项。 第 2 部分- CloudFront 故障转移配置。 第 3 部分- Amazon API Gateway HTTP API 故障转移和延迟配置。 第 4 部分- Amazon DynamoDB 全局表。 部

avatar 云原生

使用 Visual Studio IDE 使用 .NET Core 6 运行时创建简单的 AWS Lambda 函数

在这篇博客中,让我们看看如何开始使用 .NET Core 6 创建无服务器函数 (AWS Lambda) 我们要做什么? 使用 Visual Studio 创建 Lambda 函数 将函数推送到 AWS Lambda 确定测试功能的不同方法 先决条件: 1.Visual Studio 2022 社区版 2.AWS Toolkit for Visual Studio- 用于使用 Visual Stu

avatar 云原生

在 Mac 上即时提升 Docker 性能

macOS 适合运行 Docker 吗? Mac 是优秀的开发平台。它们支持 Java、Python、Ruby 和许多其他常用的开发语言。它们让开发人员打开一个 shell 并找到一个看起来很像 Linux 的环境。许多开发者都欣赏 Mac 和 Apple 以出名的用户体验。更重要的是,Mac 支持最流行的开发工具,如 VS Code、JetBrains 工具箱和......Docker——这是测

avatar 云原生