Abstract

Unlike many previously proposed solutions for domain adaptation, our algorithm does not consist of a fixed reweighting of the losses over the training sample. Instead, the reweighting depends on the hypothesis sought. The algorithm is derived from a less conservative notion of discrepancy than the DM algorithm called generalized discrepancy.

In addition to performing linear classification, SVMs can efficiently perform a non-linear classification using what is called the kernel trick, implicitly mapping their inputs into high-dimensional feature spaces.

When data are not labeled, supervised learning is not possible, and an unsupervised learning approach is required, which attempts to find natural clustering of the data to groups, and then map new data to these formed groups. The clustering algorithm which provides an improvement to the support vector machines is called support vector clustering[2] and is often used in industrial applications either when data is not labeled or when only some data is labeled as a preprocessing for a classification pass.

reference articles

Adaptation Based on Generalized Discrepancy

1 permission description

WebView can open the local file by way of url, the way of controlling this privilege is

1
2
3
4

            // Set whether to allow other local files to be read by the Js code loaded by file url
            webSettings.setAllowFileAccessFromFileURLs (false);
            / / Set whether to allow Javascript loaded through the file url can access other sources (including http, https and other sources)
            webSettings.setAllowUniversalAccessFromFileURLs (false);

The default is turned on, in Android O, permissions tightening refinement may not be the problem
The root cause of the problem is that the app being used can read the local file it can access and the loss is viewed as a local file (which can be accessed by the APP).

2 Threat scenario analysis

The threat has been used in the following scenarios

2.1 The visited webpage is modified

General WebView is mainly used to access some more changes are not suitable for native implementation of the page or style complex but unchanging page. This page is generally obtained through the http protocol (including https), if the visit to the page is hijacked or the source server is directly attacked then visit the wrong will be a threat

2.2 Low Permissions APP Get the information through high authority APP

The same system, the general competence is the same. However, some systems or applications have gained root privileges, higher authority. Low Permissions APPs can use WebView as a springboard to directly access files that can not be accessed through these applications. However, the premise is that the WebView where the page can be opened by other App. High privileges here may not be limited to accessing system-level files, but may also be some APP-specific files, such as signature files, cached keys, etc., which can be accessed by only high-privilege APPs.

Fixed

It is recommended to close both permissions

1
2
3
4

// Set whether to allow other local files to be read by the Js code loaded by the file url
WebSettings.setAllowFileAccessFromFileURLs (false);
/ / Set whether to allow Javascript loaded through the file url can access other sources (including http, https and other sources)
WebSettings.setAllowUniversalAccessFromFileURLs (false);

If there is a need for local file access, try to turn off the JavaScript protocol

1
2
3
4
5
6

// Do not allow file protocol to load JavaScript
If (url.startsWith ("file: //") {
SetJavaScriptEnabled (false);
} else {
SetJavaScriptEnabled
}

ref

WebSettings
About WebView Permission Error

Instruction

The matrix-completion problem has attracted a lot of attention, largely as a result of the celebrated Netflix competition. Two popular approaches for solving the problem are nuclear-norm- regularized matrix approximation (Candes and Tao, 2009; Mazumder et al., 2010), and maximum-margin matrix factorization (Srebro et al., 2005). These two procedures are in some cases solving equivalent problems, but with quite different algorithms. In this article we bring the two approaches together, leading to an efficient algorithm for large matrix factorization and completion that outperforms both of these. We develop a software package softImpute in R for implementing our approaches, and a distributed version for very large matrices using the Spark cluster programming environment

Ref

abs

pdf

1 Introduction

Tmall client users, how to ensure the stability of the cat is very important task, and the start-up stage of protection is one of the key part.

Tmall security model is committed to solving the problem such as the crash of the APP start-up phase, with self-healing ability, synchronous hot repair capability, is a set of start protection solution.

The origin of the Tmall security model

• Question: APP in use, and sometimes encounter online can not repair the crash crash, the user can not use APP

• Thinking:

1. Can we avoid this problem? Is there a way for the program to automatically fix the problem?

2. How can we better repair similar problems?

• in conclusion:

We need a solution that will ensure that APP starts smoothly and resolves major issues - security model

2 Design of Tmall Safe Mode

Tmall security model focuses on the start-up phase to solve the problem, from the configuration background, the client capabilities, data, testing four areas are given a unified solution, but also take into account the different APP compatibility issues

2.1 Configure the background

A unified configuration background, with gray release mechanism

2.2 Client Capability

1. In the case of APP continuous Crash with classification, no sense of self-healing ability

2. With synchronous hot repair capability

3. Ability to specify the ability to trigger a particular function

4. With the ability to register, you can easily extend the security model later

3.3 Data statistics and alarms

1. Unified data platform

2. Monitoring alarm function, allowing you to discover the problem in time

3. You can view the success rate of heat repair and other data

3.4 Quick test

1. Optimize pre-emptive testing

2. Optimization of each regression to verify the difficulty of safe mode

3.5 The Development of Tmall Security Model

Security mode has so far experienced four major versions, the function has been constantly improved, the following figure describes the main function of each version

3.6 Tmall safe mode principle

APP crash for many reasons, each APP design solutions are different, all of its abnormal errors are very difficult to capture, so we changed the way, completely from the user’s point of view what is abnormal exit, that is, playing Flag the flag

1. How to determine the abnormal exit:

• A flag value is logged when APP starts

• The flag value is cleared when the following conditions are true:

APP normally starts for 10 seconds

User exits the application normally

The user automatically switch from the foreground to the background

• If an exception occurs during the startup phase, the flag is not cleared, and the flag can be used to determine whether the client is quitting abnormally.

• Each time an exception exits, the flag value is +1

2. Hierarchical Execution Policy for Safe Mode:

• Security mode according to the size of the value of the flag to do a hierarchical implementation of the strategy, the current security is divided into two levels, continuous crash 2 times for a security model, continuous crash 3 times and above for the two security model

• Line of business can be registered in a safe mode of behavior, such as a business to empty the cache data, so that when entering a safe mode, the security model will automatically call the registration behavior, try to repair the client

• If the first-level security model can not repair APP, will enter the secondary security model, the secondary security model will restore the APP to the initial installation state, Document, Library, Cache three root directory empty

3. Hot fix execution strategy:

• Old version of the hot fix strategy: the secondary security model in the trigger

Question: 3 consecutive crashes after the trigger, in the case of problems, to open so many times APP users too little, we can not repair faster?

• New version of the repair strategy:

Hot fix from the specific level of stripping, as long as the need to find hot fix configuration, APP will be blocked simultaneously hot repair, to ensure the timeliness of repair

4. Grayscale scheme:

• Security mode to develop a simple grayscale strategy, gray, the configuration will also include the gray, the official two copies of the configuration, will also include the probability of gray

• APP according to a specific algorithm to calculate whether they meet the gray-scale conditions, if satisfied, the use of grayscale configuration, or the use of formal configuration

3.7 Thinking on the Ease of Use of the Tmall Security Model

At first, we did not particularly consider ease of use, because the first two versions are only one access to the cat, do not consider the difference; but in the docking group other APP, we find that the demand point is still a bit Big difference, but also found the lack of security model, so we increased the ease of use considerations, mainly in the following points:

1. Access cost

• Stand in the access angle, improve the document, redefine the interface, and strive to interface is simple, clear, lower access costs

2. Unified configuration background

• Easy access to the configuration information, the use of Ali cloud CDN services to build a unified configuration center, according to APP, version to configure

3. Customization

• Taking into account the different APP location and the actual needs of different security model to support the transformation of customized features, so that the access side to determine the specific behavior

• For example, the day cat that users are concerned about the main function is available, do not care whether to enter the safe mode, so do not need to display a separate prompt page, but communicate with other APP found that some APP still want to have such a prompt page to inform the user What we did

4. Grayscale mechanism

• Safe mode, the initial position is only used to solve the problem of starting crash, but in the group with other APP communication process found that they also need to not crash in the case of APP to solve the problem of direct release is still very dangerous, Gray-scale mechanism is very urgent, so we achieved in the 4.0 version of the gray-scale mechanism

5. data analysis

• Using the Group’s unified data platform to facilitate the access side to query the relevant data, while improving the security model to provide a basis

6. Quick test

• For the test (simulated continuous crash) by adding special treatment, improve test efficiency

Conclusion

Security model is currently developed to V4.0 version, has been on the line more than six months time, a good guarantee of the cat’s APP start-up security, follow-up we will continue to polish its security model to better protect the escort for the APP.

ref:

The final rescue plan - safe mode

1 Prevent static analysis

1. against the anti-compiler tools: through a large number of apk package testing can not find anti-compiled instructions or fields, these fields or instructions used in the development of the code to prevent others from decompiling methods: apktool and dex2jar tools and a large number of APK Do verification tests to find these programs can not resolve the field or code.

2. the code obfuscation technology: code development using confusing technology, making the decompile code and methods are used a, b, c letters instead of increasing the difficulty of reverse analysis crack, method android comes with proguard can be achieved.

3. NDK protection: Java development of the code is extremely easy to decompile and reverse, and the use of c / c ++ development. So libraries need to view the assembly code in order to reverse the reduction, the more difficult, methods: the use of NDK development of core code.

4. shell protection: .so file can be used to protect the compressed encryption shell, method: the use of arm-linux kernel procedures can be achieved with shell tools.

2 To prevent dynamic debugging

Check the governor, check the simulator, the direct use of SDK API function checksum can be checked.

3 Prevent re-packaging

Signature verification: Call sdk api to get signature information Write signature verification at the beginning of the program to verify whether the signature was re-modified.

Check protection: record crx xx.dex checksum, stored in a string file or stored on the network, the code to verify the comparison to ensure that the dex file has not been modified.

reference articles:

• chinese blog

Instruction

In computing, a graph database is a database that uses graph structures for semantic queries with nodes, edges and properties to represent and store data. A key concept of the system is the graph (or edge or relationship), which directly relates data items in the store. The relationships allow data in the store to be linked together directly, and in most cases retrieved with a single operation.

This contrasts with conventional relational databases, where links between data are stored in the data itself, and queries search for this data within the store and use the JOIN concept to collect the related data. Graph databases, by design, allow simple and rapid retrieval of complex hierarchical structures that are difficult to model in relational systems. Graph databases are similar to 1970s network-model databases in that both represent general graphs, but network-model databases operate at a lower level of abstraction[1] and lack easy traversal over a chain of edges.[2]

The underlying storage mechanism of graph database products varies. Some depend on a relational engine and store the graph data in a table while others use a key-value store or document-oriented database for storage, making them inherently NoSQL structures. Most[according to whom?] graph databases based on non-relational storage engines also add the concept of tags or properties, which are essentially relationships lacking a pointer to another document. This allows data elements to be categorized for easy retrieval en masse.

Retrieving data from a graph database requires a query language other than SQL, which was designed for relational databases and does not elegantly handle traversing a graph. As of 2016, no single graph query language has been universally adopted in the same fashion as SQL was for relational databases, and there are a wide variety of systems - most often tightly tied to a particular product. Some standardization efforts have taken place, leading to multi-vendor query languages like Gremlin, SPARQL, and Cypher. In addition to having query language interfaces, some graph databases are accessed through APIs.

Accessiable products:

• cayley(google)

• neo4j

• allegrograph

• flockdb(Twitter)

• infinitegraph

Reference

Graph database wiki

use neo4j

1 The protocol stack

2 SSL / TLS

2.1 Handshake Protocol

2.1.1 Initial establishment

Step1. Client Hello

1. Supported protocol versions, such as TLS version 1.0

2. A client-generated random number, which is then used to generate a “session key”

3. Support of encryption methods, such as RSA public key encryption

4. Supported compression methods

Step2. Server Hello

1. Confirm the version of the encrypted communication protocol used, such as TLS version 1.0. If the browser does not match the version supported by the server, the server closes the encrypted communication

2. A server-generated random number, which is then used to generate the “session key”

3. Confirm the encryption method used, such as RSA public key encryption

4. Server certificate

Step3. Client Key Exchange

1. A random number. The random number is encrypted with the server public key to prevent eavesdropping

2. Code change notification that the subsequent information will be transmitted using both the agreed encryption method and the key

3. The client handshake end notification indicates that the client’s handshake phase has ended. This item is also the hash value of all the content sent before, used for server validation

Step4. Server Finish

After receiving the PreMaster encrypted data from the client, the server uses the private key to decrypt the encrypted data and validate the data. The server generates Session Secret in the same way as the client. When everything is ready, It sends a ChangeCipherSpec to the client informing the client that it has switched to the negotiated cipher suite state and is ready to encrypt the data using the cipher suite and Session Secret.

2.1.2 Restore the connection

If the session is interrupted, and the next time the reconnection, as long as the client gives the number, and the server has this number of records, the two sides can re-use the existing “dialogue key” without having to re-establish trust and security channel .

2.2 Message recording protocol

Application layer data, after segmentation, compression (optional), encryption of the last packet, through TCP / IP to another segment, in another section of the reverse operation, access to clear text data

3.TLS security analysis

3.1 Man-in-the-middle attack

Proxy

3.2 strong crack

Using a key algorithm may not be secure

3.3 There are security issues in the implementation

4. Summary

HTTPS can be securely transmitted, but not with HTTPS can sit back and relax

ref

chinese blog

Jeff Moser, The First Few Milliseconds of an HTTPS Connection

Wikipedia, Transport Layer Security

StackExchange, How does SSL work?

Instruction

Apache Kylin? is an open source Distributed Analytics Engine designed to provide SQL interface and multi-dimensional analysis (OLAP) on Hadoop supporting extremely large datasets, original contributed from eBay Inc.
Apache Kylin? lets you query massive data set at sub-second latency in 3 steps.

• Identify a Star Schema on Hadoop.
• Build Cube from the identified tables.
• Query with ANSI-SQL and get results in sub-second, via ODBC, JDBC or RESTful API.

Reference

kylin

The Process of Android Boot

Recovery.img, which consists of boot_img_hdr + zImage + recovery-ramdisk. Boot_img_hd is a structure it describes a lot of important information.

This flowchart simply describes the flow of the bootloader in the bootloader. When the device power or Reboot into the bootloader will detect whether there is a special key is pressed at that time is the KeyPress in the flowchart, for example, a cell phone while pressing the camera key + volume key will enter the recovery. Some people will ask how to detect the button is pressed? Is very simple to read the value of the keyboard control register, if you are a GPIO button GPIO register. If no key is pressed, the bootloader reads the bootloader_message structure information in the misc partition.

Android Partitions Explained: boot, system, recovery, data, cache & misc

Let’s start with a list of standard internal memory partitions on Android phones and tablets. These are:

• /boot
• /system
• /recovery
• /data
• /cache
• /misc

In addition, there are the SD card partitions.

• /sdcard
• /sd-ext

Note that only /sdcard is found in all Android devices and the rest are present only in select devices. Let’s now take a look at the purpose and contents of each of these partitions.

/boot

This is the partition that enables the phone to boot, as the name suggests. It includes the kernel and the ramdisk. Without this partition, the device will simply not be able to boot. Wiping this partition from recovery should only be done if absolutely required and once done, the device must NOT be rebooted before installing a new one, which can be done by installing a ROM that includes a /boot partition.

/system

This partition basically contains the entire operating system, other than the kernel and the ramdisk. This includes the Android user interface as well as all the system applications that come pre-installed on the device. Wiping this partition will remove Android from the device without rendering it unbootable, and you will still be able to put the phone into recovery or bootloader mode to install a new ROM.

/recovery

The recovery partition can be considered as an alternative boot partition that lets you boot the device into a recovery console for performing advanced recovery and maintenance operations on it. To learn more about this partition and its contents, see the ‘About Android Recovery’ section of our guide to ClockworkMod recovery.

/data

Also called userdata, the data partition contains the user’s data – this is where your contacts, messages, settings and apps that you have installed go. Wiping this partition essentially performs a factory reset on your device, restoring it to the way it was when you first booted it, or the way it was after the last official or custom ROM installation. When you perform a wipe data/factory reset from recovery, it is this partition that you are wiping.

/cache

This is the partition where Android stores frequently accessed data and app components. Wiping the cache doesn’t effect your personal data but simply gets rid of the existing data there, which gets automatically rebuilt as you continue using the device.

/misc

This partition contains miscellaneous system settings in form of on/off switches. These settings may include CID (Carrier or Region ID), USB configuration and certain hardware settings etc. This is an important partition and if it is corrupt or missing, several of the device’s features will will not function normally.

/sdcard

This is not a partition on the internal memory of the device but rather the SD card. In terms of usage, this is your storage space to use as you see fit, to store your media, documents, ROMs etc. on it. Wiping it is perfectly safe as long as you backup all the data you require from it, to your computer first. Though several user-installed apps save their data and settings on the SD card and wiping this partition will make you lose all that data.

On devices with both an internal and an external SD card – devices like the Samsung Galaxy S and several tablets – the /sdcard partition is always used to refer to the internal SD card. For the external SD card – if present – an alternative partition is used, which differs from device to device. In case of Samsung Galaxy S series devices, it is /sdcard/sd while in many other devices, it is /sdcard2. Unlike /sdcard, no system or app data whatsoever is stored automatically on this external SD card and everything present on it has been added there by the user. You can safely wipe it after backing up any data from it that you need to save.

/sd-ext

This is not a standard Android partition, but has become popular in the custom ROM scene. It is basically an additional partition on your SD card that acts as the /data partition when used with certain ROMs that have special features called APP2SD+ or data2ext enabled. It is especially useful on devices with little internal memory allotted to the /data partition. Thus, users who want to install more programs than the internal memory allows can make this partition and use it with a custom ROM that supports this feature, to get additional storage for installing their apps. Wiping this partition is essentially the same as wiping the /data partition – you lose your contacts, SMS, market apps and settings.

With this, we conclude our tour of Android partitions. Now whenever you install a ROM or mod that requires you to wipe certain partitions before the installation, you should be in a better position to know what you’re losing and what not and thus, you’ll know what to backup and what not.

reference articles:

explation1

explation2

explation3

Fastlane Introduction

fastlane is the tool to release your iOS and Android app .

It handles all tedious tasks, like generating screenshots, dealing with code signing, and releasing your application.

Continue writting……

reference articles:

fastlane