Dataswift's Resource Repository Build Technical FAQ

Security

Documents

Data Audit and Access Control

### How long are audit logs kept for? Audit log retention is configurable on a per-cluster basis. The default duration is 1 year, however log retention could be prolonged upon request. ### Data erasure / data retention policies and procedures Customers can choose to delete data they have uploaded at any point by choosing to delete their account. The account deletion operation deletes the corresponding user's database with all data that has been uploaded. The data is only retained in encrypted backups for a reasonable amount of time to allow for recovery in case of unintentional data erasure or system failure. ### Information security and audit measures implemented at organisation level By default, the organisation does not have access to user data and the audit logs defined above to ensure that any intended or unintended access for both privileged and unprivileged users is logged and reported. The system contains implementation of automated rule checking for audit logs being enabled, disabled SSH access to production systems and minimum scope access for required users. Deployment procedures are periodically reviewed and validated by an independent party (HAT Community Foundation). ### Data back-up procedure All compute, storage and database resources are located within a single configured geographic region and distributed across multiple "Availability Zones" (independent failure zones) for resilience. Partial outages on the part of the cloud provider are handled automatically and transparently through a combination of load-balancers, automatic scaling and "hot standby" resources. In case of a failure, automated processes move data and traffic away from the affected area. Infrastructure blueprints for major outage recovery are managed via a version-control system and all data is backed up in resilient storage, with backups kept for 7 days. All data at rest as well as backups are kept in the same geographic region and encrypted. All current deployments are located within the EU.

Last updated: 5 years ago

Security: How is PDA data encrypted?

### Data at rest Data at rest is stored in two forms: - Files are stored in AWS S3 Key-Value Store. - Data is stored in AWS Relational Data Store (RDS) Database Servers. File storage is configured with server-side encryption using AES-256 encryption. Storage policy enforces any file uploaded into the storage to be encrypted. Data in RDS Servers is stored in isolated databases for each PDA owner, encrypted at rest using AES-256. All logs, backups and snapshots for a Database Server are encrypted. Database Servers’ stand-by replicas maintained for reliability are also encrypted. ### Data in transit PDA infrastructure uses industry-standard tiered network setup, segregated into three areas: - A public subnet reachable from the outside Internet. All communications are encrypted using SSL (HTTPS). Any unsecured connection is redirected to HTTPS endpoints. AWS Elastic Load Balancer (ELB) with application (HTTPS) level SSL is configured for load balancing and encrypted connection termination. - A private subnet where (HAT) Web Servers run, only reachable from inside the public subnet using a limited set of ports (all denied by default, selected ones open) - managed using a combination of explicit routing rules and firewall settings. Communication between the public subnet and the application servers is not encrypted, however it is isolated from the outside and only communication between the SSL-terminating Load Balancers and Application Servers is enabled. - A database subnet where database servers are placed, only reachable from inside the private subnet using a limited set of ports. Communication between the Application Servers and the Database Servers happens on a private, isolated network.

Last updated: 5 years ago

Do you have a separate authentication micro-server in which HAT users can authenticate without activating their personal HAT? In other words, can we have a "login with HAT" button that verifies the identity of the user and that will give us the option to authenticate a user on our side?

All the user authentication is done within their individual HAT Micro-server. It was designed like that to ensure security and decentralisation. Central authentication micro-service would become a single point of failure, obvious target for hackers and technically would give even us access into the user's HAT PDA.

Last updated: 5 years ago

Security: Is data ever stored in an unencrypted form?

Data in-use is stored in a PostgreSQL database. While the database is encrypted, the data itself is not encrypted client-side and is stored in its original form. Data may be cached by a HAT Web Server or a cluster-internal cache server in-memory for a short amount of time to enhance performance.

Last updated: 5 years ago

Security: How do you handle Server and Network Management?

### Penetration testing Third-party penetration testing occurs periodically or after significant changes to the infrastructure or as when it is deemed necessary. Automated vulnerability testing is scheduled monthly on an exact replica of the deployed cluster. In AWS-based deployments, Amazon Inspector is employed for regular testing. This automatically assesses applications for vulnerabilities or deviations from best practices. Amazon Inspector includes a knowledge base of hundreds of rules mapped to common security best practices and vulnerability definitions. Examples of built-in rules include checking for remote root login being enabled, or vulnerable software versions installed. These rules are regularly updated by AWS security researchers. The setup uses a set of third-party defined Rules Packages: - Common Vulnerabilities and Exposures: the rules in this package help verify whether the EC2 instances in your assessment targets are exposed to common vulnerabilities and exposures (CVEs). Attacks can exploit unpatched vulnerabilities to compromise the confidentiality, integrity, or availability of your service or data. The CVE system provides a reference method for publicly known information security vulnerabilities and exposures. For more information, go to ​https://cve.mitre.org/.​ - Centre for Internet Security (CIS) Benchmarks: the CIS Security Benchmarks program provides well-defined, un-biased and consensus-based industry best practices. Specifically, package Amazon Linux 2015.03 (CIS Benchmark for Amazon Linux 2014.09-2015.03, v1.1.0, Level 1 Profile) is used. ### Server software updates Servers are automatically patched nightly. To avoid disrupting system under load, the method for patching benefits from its automatic scaling setup: - Every 24 hours the size of the VM cluster serving the system is increased by 50% - Cluster monitoring tools detect when the cluster is overly provisioned (e.g. based on CPU load) and spins down a portion of the instances - Automatic scaling is set up to spin down the oldest VM instances in the cluster - Every new VM instance sets up full updates automatically on first launch Recycling servers in this manner keeps all servers patched on a daily basis. ### Firewall policy All virtual machines run inside a VPC (Virtual Private Cloud). No HAT API endpoints are exposed directly to the outside, but only to the elastic load balancer (ELB). ELB runs under a separate Security Group with ports 80 and 443 (HTTP and HTTPS respectively) exposed publicly. Database server ports are standard PostgreSQL database ports (5432), however they are not exposed publicly - only to management software inside a separate security group and running inside the private tier of the network. Port scanning is a violation of AWS TOS without explicit prior approval and is automatically detected, stopped and blocked. DDoS attack risk is mitigated through the default defence mechanisms. AWS Shield Standard is enabled by default and provides always-on network flow monitoring which inspects incoming traffic to AWS and uses a combination of traffic signatures, anomaly algorithms and other analysis techniques to detect malicious traffic in real-time. If and when the risk of DDoS attacks increases, an advanced defence mechanism is ready to be integrated. ### Intrusion detection Production deployments use an open sourced Host-Based Intrusion Detection System (HIDS) OSSEC: ​http://ossec.github.io​. HIDS is integrated with automated alerting and logging to notify administrators of security events. OSSEC actively monitors all aspects of Unix system activity with file integrity monitoring, log monitoring, root check, and process monitoring. ### Virtual Machine Segregation Building on top of AWS Xen hypervisor based virtualisation solutions and the Virtual Private Cloud network virtualisation environment, all application servers run inside separate Docker containers, isolating them from one another. Multiple Docker containers may be scheduled to run on a single Virtual Machine (VM), however VMs isolate all resources used by the application server from other cloud tenants. VMs have no control over which containers they host and containers can be moved from one VM to another in response to changes in load and resource availability. VMs are never accessed by administrative staff directly (SSH login is disabled) and are instead orchestrated through daemon applications installed to the VMs at launch: ● Application Software container orchestration tools (AWS ECS) - all interactions are recorded in centralised system logs ● Systems Manager if any remote shell command execution is required - disabled by default; if enabled, all interactions are recorded and alerts generated Database Servers (PostgreSQL) run separately, with one Database Server per VM instance and multiple databases running on the same Database Server. Each HAT maintains their data in a separate, isolated database instance with no data shared across multiple databases directly. ### Recovery timescales Recovery and scaling is virtually instantaneous in case of partial underlying infrastructure failures, transparent to the users and done without technical staff involvement. Using infrastructure blueprints major outages can be recovered from within hours of the underlying infrastructure becoming available.

Last updated: 5 years ago

Security: What is the HAT Microserver Password Policy?

### Password strength policy The HAT uses a zxcvbn-based password strength policy (published in USENIX Security'16). zxcvbn is a password strength estimator inspired by password crackers. Through pattern matching and conservative estimation, it recognises and weighs 30,000 common passwords, common names and surnames according to US census data, popular English words from Wikipedia and US television and movies, and other common patterns such as dates, repeats (aaa), sequences (abcd), keyboard patterns (qwertyuiop), and l33t speak. The policy builds on a computational password complexity score instead of rules like "passwords must contain three of {lower, upper, numbers, symbols}" for a more secure alternative. It allows many password styles to flourish so long as it detects sufficient complexity – passphrases are rated highly given enough uncommon words, keyboard patterns are ranked based on length and number of turns, and capitalisation adds more complexity when it's unpredictable. Specifically, the HAT is configured to require the estimated number of guesses to find the password to be at least 10^10. ### What encryption is used to secure username / password login Passwords are only stored in their Bcrypt-hashed form. HTTPS is enforced for all operations including password-based login. Users are strongly advised not to enter their password anywhere but their personal HAT page with a valid SSL certificate. Passwords are required to be transmitted in request body or headers to avoid unintended capture in request logs. A user password or its hash is never shared with other entities; other entities such as platform providers (for limited management operations) and data clients (for data requests) use separate credentials, configured at the time of provisioning a HAT (in the former case) or provisioned by the platform provider and with limited rights at a later stage.

Last updated: 5 years ago