Mercurial > public > tweet-analysis

annotate dependencies/python/aws_controller.py @ 8:6541622b6127

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
add tweet analysis method
author Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
date Fri, 17 Sep 2021 21:10:02 +0200
parents
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
8
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
1 import boto3
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
2 import base64
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
3 import json
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
4 from botocore.exceptions import ClientError
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
5
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
6
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
7 class AwsSecretsManager:
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
8 @staticmethod
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
9 def get_secret(secret_name, region_name='eu-west-2'):
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
10 """
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
11 Get Secret Keys from AWS Secrets Manager
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
12 :return:
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
13 """
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
14
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
15 # Create a Secrets Manager client
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
16 session = boto3.session.Session()
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
17 client = session.client(
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
18 service_name='secretsmanager',
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
19 region_name=region_name
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
20 )
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
21
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
22 try:
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
23 get_secret_value_response = client.get_secret_value(SecretId=secret_name)
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
24 except ClientError as e:
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
25 if e.response['Error']['Code'] == 'DecryptionFailureException':
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
26 # Secrets Manager can't decrypt the protected secret text using the provided KMS key.
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
27 raise e
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
28 elif e.response['Error']['Code'] == 'InternalServiceErrorException':
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
29 # An error occurred on the server side.
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
30 raise e
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
31 elif e.response['Error']['Code'] == 'InvalidParameterException':
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
32 # You provided an invalid value for a parameter.
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
33 raise e
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
34 elif e.response['Error']['Code'] == 'InvalidRequestException':
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
35 # You provided a parameter value that is not valid for the current state of the resource.
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
36 raise e
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
37 elif e.response['Error']['Code'] == 'ResourceNotFoundException':
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
38 # AWS Can't find the resource that you asked for.
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
39 raise e
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
40 else:
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
41 # Decrypts secret using the associated KMS CMK.
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
42 # Depending on whether the secret is a string or binary, one of these fields will be populated.
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
43 if 'SecretString' in get_secret_value_response:
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
44 secret = get_secret_value_response['SecretString']
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
45 else:
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
46 secret = base64.b64decode(get_secret_value_response['SecretBinary'])
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
47
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
48 return json.loads(secret)
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
49
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
50
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
51 class AwsComprehend:
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
52
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
53 @staticmethod
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
54 def get_sentiment(tweets):
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
55 """
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
56 :param tweets: list (string), required
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
57 :return: dict
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
58 """
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
59 # Create a Comprehend client
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
60 session = boto3.session.Session()
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
61 comprehend = session.client(
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
62 service_name='comprehend',
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
63 region_name='eu-west-2'
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
64 )
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
65
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
66 response = comprehend.batch_detect_sentiment(
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
67 TextList=tweets,
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
68 LanguageCode='en'
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
69 )
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
70
6541622b6127 add tweet analysis method
Dennis Concepcion Martin <dennisconcepcionmartin@gmail.com>
parents:
diff changeset
71 return response