Posts

Showing posts from February, 2018

Sensitive personal data

Besides personal data there is sensitive personal data, defined as information about a person’s:


Racial or ethnic originPolitical opinionsReligious or philosophical beliefsTrade union membershipHealth dataSex life or sexual orientationPast or spent criminal convictions

The AWS Cloud Practitioner Essentials course is intended for individuals who seek an overall understanding of the AWS Cloud, independent of specific technical roles.

It provides a detailed overview of cloud concepts, AWS services, security, architecture, pricing, and support.

Load your directories into MySQL table

$ ls -l | grep '^d' | awk '{print $9}'| tee ~/directories.txt
mysql>load data local infile 'directories.txt' into table transit.img_directories;

Display or list directories only in Linix

$ ls -d */

DKIM Example

default._domainkey.subdomain.i88.ca. Selector Record default._domainkey.subdomain.i88.ca IN TXT “k=rsa\; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAjLdW5tYf5P33X59q9GkzdHWRiPDxveUd+5orhBvLxtyga3Nc7iHeGrVTvaHwefTK7egx4J4cmghtFm6gY1fz5PzwFl0eXMTGZK8kukLwjTuCJ7BJjRkdNR/AKENkcEeDVzvq25U1W+W2JWVfLTKqvU31TwBhF/gLUNU2j7ioiCBD5WspOa42UkIzNf8dKMLOfAwwgKMo92JNP5dGw4TYCR+R7lup+zPGiriX0D8DTuU23966bB14kZMJfyhu1JOQ58j1SKlVTia+cVyYgN8DaSX2tiNvtS6udKXxe8W+JoqBNP2y+K95M14E2setabR9uIF1lfrnoxAzTrJh+i1sHQIDAQAB”

After setting up your DNS record, you can check the status of the Selector record at http://www.dnswatch.info/dns/dnslookup?la=en&host=default._domainkey.subdomain.i88.ca&type=TXT&submit=Resolve

Montreal GCP region – public launch

The first Canadian GCP region, northamerica-northeast1, is now open in Montreal, Quebec, with 3 availability zones.

Switch from Tape to Cloud for Backups

AWS Storage Gateway enables your on-premise applications, including backup tools, to seamlessly use AWS cloud storage. This hybrid cloud storage service can be set up as a virtual tape library (VTL) to help you avoid the headaches of managing tape media, libraries and archive services.

Benefits of using Storage Gateway as a VTL:



Non-disruptive way to switch to cloud backups

Keep trusted backup software in place

Reduce manual effort for backups

Eliminate cost spikes of tape system replacement

Safety of Amazon S3 and Amazon Glacier (designed for 99.999999999% durability)

MySQL mysqlbackup partial backup example

In the below command all the tables from database i88ca are exclued: # ./mysqlbackup --user root --socket=/tmp/mysql.sock  --with-timestamp --backup-dir=/root/backups/   --exclude-tables="i88ca.*"  backup For excluding multiple tables/database, you could pass multiple values separated by "|" # ./mysqlbackup --user root --socket=/tmp/mysql.sock  --with-timestamp --backup-dir=/root/backups/   --exclude-tables="i88ca.*|hdgem.*"  backup The --include-tables options works the same way as --exclude-tables, however when you specify both the options in the same command --include-tables will take precedence.

Datasets

Minimize the overall size of MySQL InnoDB data files

To minimize the overall size of InnoDB data files, consider enabling the MySQL configuration option innodb_file_per_table. This option can minimize data size for InnoDB tables in several ways:It prevents the InnoDB system tablespace from ballooning in size, allocating disk space that can afterwards only be used by MySQL. For example, sometimes huge amounts of data are only needed temporarily, or are loaded by mistake or during experimentation. Without theinnodb_file_per_table option, the system tablespace expands to hold all this data, and never shrinks afterward. It immediately frees the disk space taken up by an InnoDB table and its indexes when the table is dropped or truncated. Each table and its associated indexes are represented by a .ibd file that is deleted or emptied by these DDL operations. It allows unused space within a .ibd file to be reclaimed by the OPTIMIZE TABLE statement, when substantial amounts of data are removed or indexes are dropped. It enables partial backups whe…

DMARC is a type of email authentication protocol that leverages the widely used SPF and DKIM protocols to improve a sender’s understanding of how their email in circulation is processed.

Email claiming to be from their domain is analyzed by receiving organizations and a digest of acceptance/failures is sent back to the sender. DMARC is used to reduce spam and fraudulent email by giving senders information on what recipients see. DMARC stands for Domain-based Message Authentication, Reporting & Conformance.

DKIM stands for DomainKeys Identified Mail which was designed to help ISPs prevent malicious email senders by validating email from specific domains.

Spoofers and phishers can be sending email to unwitting recipients by purporting to be from a trusted brand or sender. By "signing" email with DKIM, legitimate senders can label which domains belong to them, and by doing so, empower ISPs to block email streams that have not been properly authenticated using DKIM.

As one of the most popular email authentication methodologies, it works by using cryptographic technology that adds a digital signature to your message header. This DKIM signature validates and authorizes your domain name in the eyes of the receiver. The DKIM signature is created using a unique string of characters stored as a public key.

When your email is received, the public key is retrieved through the DNS and decrypted by the receiver to allow them to confidently verify the identity of your domain.

While DKIM authentication is an email best practice, it's important to understand that a DKIM signature is limited in scope. It does not verify content or tell the …

Setup Chinese Input Method in Ubuntu 17.10 Artful

sudo apt install ibus-sunpinyin

After either of those commands do not forget to restart the IBus daemon

ibus restart

You might need to logout and login again.

  First, go to “Language Support”, make sure the “Chinese(simplified)” or “Chinese Traditional”(or both) is installed completely.

    Go to “Input Method”, make sure you select “ibus” as the selected user configuration. An user logout may be required.

    Go to “Settings -> Region & Language”, in the “Input Sources” section, add the “Chinese(Intelligent Pinyin)” input method.


MySQL InnoDB Lock Modes for impatients

InnoDB Lock Modes
InnoDB implements standard row-level locking where there are two types of locks, shared (S) locks and exclusive (X) locks.
A shared (S) lock permits the transaction that holds the lock to read a row. An exclusive (X) lock permits the transaction that holds the lock to update or delete a row. Intention LocksAdditionally, InnoDB supports multiple granularity locking which permits coexistence of record locks and locks on entire tables. To make locking at multiple granularity levels practical, additional types of locks called intention locks are used.  There are two types of intention locks used in InnoDB (assume that transaction T has requested a lock of the indicated type on table t): Intention shared (IS): Transaction

MySQL High Availability and Scalability - Replication

MySQL Replication enables users to cost-effectively deliver application performance, scalability and high availability. Many of the world's most trafficked web properties like eBay, Facebook, Tumblr, Twitter and YouTube rely on MySQL Replication to elastically scale-out beyond the capacity constraints of a single instance, enabling them to serve hundreds of millions of users and handle exponential growth.

By mirroring data between instances, MySQL replication is also the most common approach to delivering High Availability (HA) for MySQL databases. In addition, the MySQL replication utilities can automatically detect and recover from failures, allowing users to maintain service in the event of outages or planned maintenance.

With the release of MySQL 5.6, a number of enhancements have been made to MySQL Replication, delivering higher levels of data integrity, performance, automation and application flexibility.

With the introduction of GTIDs and MySQL Replication Utilities2 in MySQ…

Fixed: MySQLSyntaxErrorException: incompatible with sql_mode=only_full_group_by

MySQLSyntaxErrorException: SELECT list is not in GROUP BY clause and contains nonaggregated column which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by

SET @@sql_mode=(SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY',''));

Employment Declines in January in Canada

With 88,000 Fewer Jobs

The location of the Apache configuration file

On most systems if you installed Apache with a package manager, or it came preinstalled, the Apache configuration file is located in one of these locations:

/etc/apache2/httpd.conf
/etc/apache2/apache2.conf
/etc/httpd/httpd.conf
/etc/httpd/conf/httpd.conf

Both the client and the server of MySQL have their own max_allowed_packet variable

The largest possible packet that can be transmitted to or from a MySQL 5.7 server or client is 1GB.
When a MySQL client or the mysqld server receives a packet bigger than max_allowed_packet bytes, it issues an ER_NET_PACKET_TOO_LARGE error and closes the connection. With some clients, you may also get a Lost connection to MySQL server during query error if the communication packet is too large.

Both the client and the server have their own max_allowed_packet variable, so if you want to handle big packets, you must increase this variable both in the client and in the server.

You can also use an option file to set max_allowed_packet. For example, to set the size for the server to 16MB, add the following lines in an option file:


[mysqld]
max_allowed_packet=16M

KEY rollover

All cryptographic keys have a life cycle that can represented by states:
Generated == the key is created but only the “owner” knows of its properties.
Published == the key has been made public either as a public key or a hash of it.
Active == the key is in use
Retired == the has been withdrawn from service but is still published
Revoked == they key has been marked as not to be trusted ever again.
Removed == taken out of publication

Different keys move through the states in different ways depending on the usage, for example some keys are never revoked, just removing them is sufficient.

CSV applications and tools

xls2csv - reads MS-Excel file and puts its content as comma-separated data on standard output
csvkit is a suite of utilities for converting to and working with CSV, the king of tabular file formats. csvkit is to tabular data what the standard Unix text processing suite (grep, sed, cut, sort) is to text.

in2csv will convert a variety of common file formats, including xls, xlsx and fixed-width into CSV format.

All the csvkit utilities work standard input and output. Any utility can be piped into another and into another and then at some point down the road redirected to a file. In this way they form a data processing “pipeline” of sorts, allowing you to do non-trivial, repeatable work without creating dozens of intermediary files. CSV Utility programs in Linux:column (-s to change the delimiter character(s))cut (-d to change the delimiter character)paste (-d to change the delimiter character(s))join (-t to change the delimiter character)sort (-t to change the delimiter character)uniq (-f to…