天道酬勤

天下事有难易乎?为之,则难者亦易矣;不为,则易者亦难矣。

0%

JDBC

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;

String driverClassName = "com.mysql.jdbc.Driver";
String url = "jdbc:mysql://127.0.0.1:3306/test?useUnicode=true&characterEncoding=UTF-8&autoReconnect=true&useSSL=false";
String user = "user01";
String password = "user01";
	
Class.forName(driverClassName).newInstance();
Connection con = DriverManager.getConnection(url, user, password);

// 插入 
String sql = "insert into user(name, password, age) values (?, ?, ?) ;";
PreparedStatement pst = con.prepareStatement(sql);
pst.setString(1, name);
pst.setString(2, password);
pst.setInt(3, age);
num = pst.executeUpdate();
pst.close();
con.close();			
			
// 查询
String sql = "select count(*) as count from table_user ;";
PreparedStatement pst = secondCon.prepareStatement(sql);
ResultSet rs = pst.executeQuery();
while (rs.next()) {
	// TODO
}			
rs.close();
pst.close();
con.close();

jdbc-pool

Tomcat 在 7.0 以前的版本都是使用 commons-dbcp 做为连接池的实现,但是 dbcp 饱受诟病,原因有:

dbcp 是单线程的,为了保证线程安全会锁整个连接池
dbcp 性能不佳
dbcp 太复杂,超过 60 个类
dbcp 使用静态接口,在 JDK 1.6 编译有问题
dbcp 发展滞后

因此很多人会选择一些第三方的连接池组件,例如 c3p0 , bonecp, druid (@wenshao ) 等。

为此,Tomcat 从 7.0 开始引入一个新的模块:Tomcat jdbc pool

tomcat jdbc pool 近乎兼容 dbcp ,性能更高
异步方式获取连接
tomcat jdbc pool 是 tomcat 的一个模块,基于 tomcat JULI,使用 Tomcat 的日志框架
使用 javax.sql.PooledConnection 接口获取连接
支持高并发应用环境
超简单,核心文件只有8个,比 c3p0 还
更好的空闲连接处理机制
支持 JMX
支持 XA Connection

tomcat jdbc pool 的优点远不止这些,详情请看 官网

tomcat jdbc pool 可在 Tomcat 中直接使用,也可以在独立的应用中使用。

在独立的应用中使用

import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.Statement;

import org.apache.tomcat.jdbc.pool.DataSource;
import org.apache.tomcat.jdbc.pool.PoolProperties;

public class SimplePOJOExample {

  public static void main(String[] args) throws Exception {
	  PoolProperties p = new PoolProperties();
	  p.setUrl("jdbc:mysql://localhost:3306/mysql");
	  p.setDriverClassName("com.mysql.jdbc.Driver");
	  p.setUsername("root");
	  p.setPassword("password");
	  p.setJmxEnabled(true);
	  p.setTestWhileIdle(false);
	  p.setTestOnBorrow(true);
	  p.setValidationQuery("SELECT 1");
	  p.setTestOnReturn(false);
	  p.setValidationInterval(30000);
	  p.setTimeBetweenEvictionRunsMillis(30000);
	  p.setMaxActive(100);
	  p.setInitialSize(10);
	  p.setMaxWait(10000);
	  p.setRemoveAbandonedTimeout(60);
	  p.setMinEvictableIdleTimeMillis(30000);
	  p.setMinIdle(10);
	  p.setLogAbandoned(true);
	  p.setRemoveAbandoned(true);
	  p.setJdbcInterceptors(
		"org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;"+
		"org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer");
	  DataSource datasource = new DataSource();
	  datasource.setPoolProperties(p);

	  Connection con = null;
	  try {
		con = datasource.getConnection();
		Statement st = con.createStatement();
		ResultSet rs = st.executeQuery("select * from user");
		int cnt = 1;
		while (rs.next()) {
			System.out.println((cnt++)+". Host:" +rs.getString("Host")+
			  " User:"+rs.getString("User")+" Password:"+rs.getString("Password"));
		}
		rs.close();
		st.close();
	  } finally {
		if (con!=null) try {con.close();}catch (Exception ignore) {}
	  }
  }

}

异步获取连接

Connection con = null;
try {
  Future<Connection> future = datasource.getConnectionAsync();
  while (!future.isDone()) {
      System.out.println("Connection is not yet available. Do some background work");
      try {
	  Thread.sleep(100); //simulate work
      }catch (InterruptedException x) {
	  Thread.currentThread().interrupted();
      }
  }
  con = future.get(); //should return instantly
  Statement st = con.createStatement();
  ResultSet rs = st.executeQuery("select * from user");
<dependency>  
	<groupId>org.apache.tomcat</groupId>  
	<artifactId>tomcat-jdbc</artifactId>  
	<version>7.0.29</version>  
</dependency>  
<dependency>  
	<groupId>org.apache.tomcat</groupId>  
	<artifactId>tomcat-juli</artifactId>  
	<version>7.0.29</version>  
</dependency>  
<!-- https://mvnrepository.com/artifact/org.apache.tomcat/tomcat-jdbc -->
<dependency>
    <groupId>org.apache.tomcat</groupId>
    <artifactId>tomcat-jdbc</artifactId>
    <version>8.5.15</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.tomcat/tomcat-juli -->
<dependency>
    <groupId>org.apache.tomcat</groupId>
    <artifactId>tomcat-juli</artifactId>
    <version>8.5.15</version>
</dependency>

References

JDBC
[1] http://www.yiibai.com/jdbc/jdbc_quick_guide.html
[2] http://www.cnblogs.com/DreamDrive/p/5757693.html
[3] http://blog.csdn.net/whucyl/article/details/20838079
[4] http://javastudyeye.iteye.com/blog/835448

jdbc-pool
[1] https://stackoverflow.com/questions/24559468/neo4j-jdbc-connection-pool
[2] http://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html
[3] https://www.oschina.net/question/12_36910
[4] https://github.com/alibaba/druid

Java解析xml

遇到一个问题,要解析一个xml,网上搜了搜,都说有4中方式,试了试dom解析,感觉解析的时候开发效率太低,忽然想到Jsoup,然后就用了第5种方式Jsoup解析XML。
用Jsoup解析XML,开发效率确实是高,但是运行效率太低了。解析一个10K左右的xml要0.2s左右。300万的xml文件要解析到什么时候呀。
然后试了试Dom解析xml,效率提高了不少,解析一个10K左右的xml 0.05s左右,效率提高的不少。
当然,还有其他3种方式解析。知道Sax解析时占用内存小,可能会快一点,但是着急处理文件,暂时没有测试。

DOM SAX JDOM DOM4J Jsoup


import javax.xml.parsers.DocumentBuilderFactory;

import java.io.*;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;

import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.ParserConfigurationException;

import com.xxxx.usdp.odk.common.file.FileUtil;
import com.xxxx.usdp.xxxx.poc.yuyin.entity.XmlEntity;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.w3c.dom.*;
import org.xml.sax.SAXException;

/**
 * DOM方式解析xml
 *
 * @version V1.0
 */

public class DomParserXml {
    
    private static final Logger log = LoggerFactory.getLogger(DomParserXml.class);
    
    /**
     * 测试解析
     */
    @Test
    public void testPaser(){
        String xmlFilePath = "src/main/resources/2018010109013384362390728_1522286730680.xml";
        xmlFilePath = "C:\\home\\user1\\xxxx\\2018010109000672062385406";
        //xmlFilePath = "src/main/resources/test.xml";
        XmlEntity xmlEntity = parserXml(xmlFilePath);
        log.info("xml数据:\r\n{}", xmlEntity);
    }
    
    /**
     * 批量解析
     */
    @Test
    public void batchParser(){
        String filePath = "D:\\data\\210_1\\210_test";
        String outDirPath = "D:\\data\\210_1\\210_201801_result";
        File outDir = new File(outDirPath);
        if(!outDir.exists()){
            outDir.mkdirs();
        }
        
        File dir = new File(filePath);
        File[] files = dir.listFiles();
        int length = files.length;
        for(int i = 0; i < 1000; i++){
            File f = files[i];
            XmlEntity xmlEntity = parserXml(f.getAbsolutePath());
            try {
                FileUtil.writeStringToFile(xmlEntity.getMix(), outDirPath+"/"+xmlEntity.getFileName()+".txt");
            } catch (IOException e) {
                log.error("写文件出错 {}", e.toString());
            }
        }
    }
    
    /**
     * 把xml解析成对话格式
     *
     * @param xmlFilePath
     */
    public static XmlEntity parserXml(String xmlFilePath) {
        return parserXml(new File(xmlFilePath));
    }
    
    
    /**
     * 把xml解析成对话格式
     *
     * @param f
     */
    public static XmlEntity parserXml(File f) {
        
        long t1 = System.currentTimeMillis();
        long t2 = 0;
        //1、创建一个DocumentBuilderFactory的对象
        DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
        //2、创建一个DocumentBuilder的对象
        Document document = null;
        XmlEntity xmlEntity = new XmlEntity();
    
        try {
            //创建DocumentBuilder对象
            DocumentBuilder db = dbf.newDocumentBuilder();
            //3、通过DocumentBuilder对象的parser方法加载books.xml文件到当前项目下
            /*注意导入Document对象时,要导入org.w3c.dom.Document包下的*/
            //传入文件名可以是相对路径也可以是绝对路径
            //document = db.parse(xmlFilePath);
            document = db.parse(f);
            xmlEntity.setFileName(f.getName().replace(".xml", ""));
            
            t2 = System.currentTimeMillis();
            log.info("读文件用时{}s", 1.0*(t2-t1)/1000);
        } catch (ParserConfigurationException e) {
            log.error("Dom解析Xml出错 {}", e.toString());
        } catch (SAXException e) {
            log.error("Dom解析Xml出错 {}", e.toString());
        } catch (IOException e) {
            log.error("Dom解析Xml出错 {}", e.toString());
        }
    
        
        Element instance = (Element) document.getElementsByTagName("instance").item(0);
        // 文件保存地址
        String waveuri = instance.getAttribute("waveuri");
        log.debug("waveuri:{}",waveuri);
        xmlEntity.setWaveuri(waveuri);
        String duration = instance.getAttribute("duration");
        log.debug("duration:{}",duration);
        xmlEntity.setDuration(duration);
        
        NodeList subjectNodes = document.getElementsByTagName("subject");
        if(subjectNodes == null || subjectNodes.getLength() < 2){
            log.error("文件格式错误,subject节点个数小于2个");
            return null;
        }
        log.debug("subject节点个数:{}", subjectNodes.getLength());
        
        
        /** 处理正文和时间片 */
        Element subject1 = (Element) subjectNodes.item(1);
        
        NodeList channels = subject1.getElementsByTagName("channel");
        log.debug("channels 节点个数:{}", channels.getLength());
        
        // channel0 n0
        Element c1 = (Element) channels.item(0);
        String tagname = c1.getTagName();
        log.debug("tagname:{}" ,tagname);
        Element textElementA = (Element) c1.getElementsByTagName("text").item(0);
        Element timeElementA = (Element) c1.getElementsByTagName("time").item(0);
        String textA = textElementA.getTextContent().trim();
        log.debug("textA:|{}|", textA);
        xmlEntity.setN0(textA);
        String timeA = timeElementA.getTextContent().trim();
        log.debug("timeA:|{}|", timeA);
        String[] textArrayA = textA.split(" ");
        String[] timeArrayA = timeA.split(" ");
        int textLengthA = textArrayA.length;
        log.debug("textLengthA:{}", textLengthA);
    
    
        // channel1  n1
        Element c2 = (Element) channels.item(1);
        String tagname2 = c2.getTagName();
        log.debug("tagname2:{}" ,tagname2);
        Element textElementB = (Element) c2.getElementsByTagName("text").item(0);
        Element timeElementB = (Element) c2.getElementsByTagName("time").item(0);
        String textB = textElementB.getTextContent().trim();
        log.debug("textB:|{}|", textB);
        xmlEntity.setN1(textB);
        String timeB = timeElementB.getTextContent().trim();
        log.debug("timeB:|{}|", timeB);
    
        String[] textArrayB = textB.split(" ");
        String[] timeArrayB = timeB.split(" ");
        int textLengthB = textArrayB.length;
        log.debug("textLengthB:{}", textLengthB);
    
        String n0 = "n0";
        String n1 = "n1";
        List<TimeTextEntity> timeTextList = new ArrayList<>(textLengthA +textLengthB);
        
        if(textLengthA > 1){
            // A
            for(int i = 0; i < textLengthA; i++){
                // 一个词语
                String oneTerm = textArrayA[i];
                // 时间片
                String oneTime = timeArrayA[i];
                String[] timeArraySub = oneTime.split(",");
                int start = Integer.parseInt(timeArraySub[0]);
                int end = Integer.parseInt(timeArraySub[1]);
                TimeTextEntity t = new TimeTextEntity(start, end, oneTerm, n0);
                timeTextList.add(t);
            }
        }
    
    
        if(textLengthB >1){
            // B
            for(int i =0; i <textLengthB; i++){
                // 一个词语
                String oneTerm = textArrayB[i];
                // 时间片
                String oneTime = timeArrayB[i];
                String[] timeArraySub = oneTime.split(",");
                int start = Integer.parseInt(timeArraySub[0]);
                int end = Integer.parseInt(timeArraySub[1]);
                TimeTextEntity t = new TimeTextEntity(start, end, oneTerm, n1);
                timeTextList.add(t);
            }
        }
    
        long t4 = System.currentTimeMillis();
        // 升序
        Collections.sort(timeTextList, new Comparator<TimeTextEntity>() {
            @Override
            public int compare(TimeTextEntity o1, TimeTextEntity o2) {
                return new Integer(o1.getStart()).compareTo(o2.getStart());
            }
        });
        long t5 = System.currentTimeMillis();
        log.info("排序用时{}s", 1.0*(t5-t4)/1000);
    
        int allCount = timeTextList.size();
        StringBuilder sb = new StringBuilder();
        String flag = null;
        for(int i =0; i < allCount; i++){
        
            log.debug("{}  {}", i, timeTextList.get(i));
        
            TimeTextEntity entity = timeTextList.get(i);
            String who = entity.getWho();
            if(who.equals(flag)){
                sb.append(entity.getText());
                sb.append(" ");
            }else{
                sb.append("\r\n");
                flag = who;
                sb.append(flag);
                sb.append(" : ");
                sb.append(entity.getText());
                sb.append(" ");
            }
        
        
        } // end for
        
        xmlEntity.setMix(sb.toString());
        
    
        long t3 = System.currentTimeMillis();
        log.info("解析用时{}s", 1.0*(t3-t2)/1000);
    
        log.info("总共用时{}s", 1.0*(t3-t1)/1000);
        
        log.debug("对话:{}", sb);
    
        return xmlEntity;
    }
    
    
}


/**
 * 时间段对象<br>
 */
class TimeEntity{
    
    private int start;
    private int end;
    
    public TimeEntity(){
    
    }
    
    public TimeEntity(int start, int end){
        this.start = start;
        this.end = end;
    }
    
    public int getStart() {
        return start;
    }
    
    public void setStart(int start) {
        this.start = start;
    }
    
    public int getEnd() {
        return end;
    }
    
    public void setEnd(int end) {
        this.end = end;
    }
    
    @Override
    public String toString() {
        return "TimeEntity{" + "start='" + start + '\'' + ", end='" + end +
            '\'' + '}';
    }
    
}

/**
 *
 */
class TimeTextEntity{
    
    private int start;
    private int end;
    private String text;
    /** n0 n1 */
    private String who;
    
    public TimeTextEntity(){
    
    }
    
    public TimeTextEntity(int start, int end, String text, String who){
        this.start = start;
        this.end = end;
        this.text = text;
        this.who = who;
    }
    
    public int getStart() {
        return start;
    }
    
    public void setStart(int start) {
        this.start = start;
    }
    
    public int getEnd() {
        return end;
    }
    
    public void setEnd(int end) {
        this.end = end;
    }
    
    public String getText() {
        return text;
    }
    
    public void setText(String text) {
        this.text = text;
    }
    
    public String getWho() {
        return who;
    }
    
    public void setWho(String who) {
        this.who = who;
    }
    
    @Override
    public String toString() {
        return "TimeTextEntity{" + "start=" + start + ", end=" + end + ", " +
            "text='" + text + '\'' + ", who='" + who + '\'' + '}';
    }
}

java学习

J2SE的架构

阅读全文 »

  对决策树的学习 以及 决策树在文本分类中的应用。

(1) 生活实例

  通俗来说,决策树分类的思想类似于找对象。现想象一个女孩的母亲要给这个女孩介绍男朋友,于是有了下面的对话:

  女儿:多大年纪了?
  母亲:26。
  女儿:长的帅不帅?
  母亲:挺帅的。
  女儿:收入高不?
  母亲:不算很高,中等情况。
  女儿:是公务员不?
  母亲:是,在税务局上班呢。
  女儿:那好,我去见见。

  这个女孩的决策过程就是典型的分类树决策。相当于通过年龄、长相、收入和是否公务员
对将男人分为两个类别:见和不见。

生活实例1

阅读全文 »

本文主要介绍有关MySQL的内容,包括一些常用配置,常见问题。根据个人使用经验总结,希望可以帮到大家。

MySQL配置

下面是我的配置

[client]

port=3306

# utf8mb4 is a superset of utf8
default-character-set = utf8mb4


[mysql]

# utf8mb4 is a superset of utf8
default-character-set = utf8mb4


# SERVER SECTION
# ----------------------------------------------------------------------
#
# The following options will be read by the MySQL Server. Make sure that
# you have installed the server correctly (see above) so it reads this 
# file.
#
[mysqld]

# utf8mb4 is a superset of utf8
character-set-server=utf8mb4


#collation-server=utf8mb4_unicode_ci 
#collation-server=utf8_general_ci

#character-set-client-handshake = FALSE

#init_connect='SET NAMES utf8mb4'

# mkdir for every database
innodb_file_per_table=1

# ignore lowercase
lower_case_table_names=1

# all import biggest 1024M file to mysql
max_allowed_packet=1024M

# The TCP/IP Port the MySQL Server will listen on
port=3306

#log

# Binary Log
log-bin=mysql-bin
binlog-format=ROW 
server_id=1

# if query_time > 1s sql will log
long_query_time=1
# if query is slow, query will log   version 5.6
slow-query-log=1
slow-query-log-file = /usr/local/mysql/log/slow_query.log
# slow-query-log-file=c:/professionsofware/mysql/log/slow_query.log
# version5.0  log-slow-queries=c:/professionsofware/mysql/log/slow_query.log

# log all query  version5.6
general_log=ON
general_log_file = /usr/local/mysql/log/all_query.log
#general_log_file=c:/ProfessionSofware/MySQL/log/all_query.log
#version5.0 log=c:/ProfessionSofware/MySQL/log/all_query.log
# log error 
#log-error=c:/professionsofware/mysql/log/mysql_error.log

#Path to installation directory. All paths are usually resolved relative to this.
#basedir="C:/ProfessionSofware/MySQL/MySQLServer5.6/"
basedir=/usr/local/mysql 

#Path to the database root
#datadir="C:/ProgramData/MySQL/MySQL Server 5.6/Data/"
datadir=/usr/local/mysql/data

# The default character set that will be used when a new schema or table is
# created and no character set is defined
#character-set-server=gbk

# The default storage engine that will be used when create new tables when
default-storage-engine=INNODB

# The default storage engine that will be used for temporary tables
default-tmp-storage-engine=INNODB
阅读全文 »

原来在学会一些东西的时候总喜欢总结,做一些笔记。刚开始用txt做笔记。使用Git来管理版本。后来很多需要图片的,发现txt不方便,就开始用word。但是在用git提交的时候就发现word的不方便了。而且word不是纯文本文件,在其他电脑或者手机上打开不太方便。后来发现了markdown,就是用markdown来写博客。

在写博客前,在网上搜了好多写博客的方法。最后发现使用markdown格式的比较多。而且最近大家都用github + hexo来搭建自己的博客,教程也比较多,所以自己也想搭一个试试。

(1) 安装软件

(1.1) 安装git

Git官网下载,或者可以在百度下载。下载完以后双击运行。一直点next,直到完成安装。

通过 git --version 验证是否安装正确以及查看版本。

(1.2) 安装Node.js

 在Node.js官网下载,或者在百度下载。推荐使用zip包,解压完配置一下就能用。

下载完解压到一个目录,解压文件到 D:\ProfessionalSoftWare\Node , 并在解压后的目录下建立 node_global和node_cache (node_global: npm全局安装路径 node_cache: npm全局缓存路径)

新建环境变量 NODE_PATH = D:\ProfessionalSoftWare\Node\node-v10.16.0-win-x64
修改环境变量 PATH 增加 %NODE_PATH%;%NODE_PATH%\node_global;

通过 node -v 验证是否安装正确以及查看版本。

注意:npm其实是Node.js的包管理工具(package manager),刚开始一直不知道nmp和Node.js是什么关系,晕了半天

阅读全文 »

(1) SpringBoot以JSON格式返回对象

<dependency>
	<groupId>com.fasterxml.jackson.core</groupId>
	<artifactId>jackson-core</artifactId>
	<version>${jackson.version}</version>
</dependency>
 
<dependency>
	<groupId>com.fasterxml.jackson.core</groupId>
	<artifactId>jackson-databind</artifactId>
	<version>${jackson.version</version>
</dependency>
 
<dependency>
	<groupId>com.fasterxml.jackson.core</groupId>
	<artifactId>jackson-annotations</artifactId>
	<version>${jackson.version</version>
</dependency>

(2) springboot hot deploy

  1. 添加对应jar包

  2. 添加插件

  3. 配置idea

  4. mvn dependencies 添加以下依赖

    <!-- add hot deployment -->
    <dependency>
    	<groupId>org.springframework.boot</groupId>
    	<artifactId>spring-boot-devtools</artifactId>
    	<optional>true</optional>
    	<scope>runtime</scope>
    </dependency>
  5. build的时候添加以下plug

    <plugin>
    	<!-- hot deployment config -->
    	<groupId>org.springframework.boot</groupId>
    	<artifactId>spring-boot-maven-plugin</artifactId>
    	<configuration>
    		<fork>true</fork>
    	</configuration>
    </plugin>
  6. 配置idea

  7. 1 File -> Settings -> Compiler 勾选 Build Project automatically

  8. 2 双击 Shift,输入 registry , 点击 Registry.. ,勾上 compiler.automake.allow.when.app.running

Allow auto-make to start even if developed application is currently running. Note that automatically started make may eventually delete some classes that are required by the application.

References

[1] Spring Boot Reference Documentation
[2] using-spring-boot
[3] spring-boot
[4] spring.io
[5] Spring Boot 学习笔记:以JSON格式返回对象
[6] springboot+idea热部署(自动刷新)
[7] spring-boot 速成(2) devtools之热部署及LiveReload

Welcome to Hexo! This is your very first post. Check documentation for more info. If you get any problems when using Hexo, you can find the answer in troubleshooting or you can ask me on GitHub.

Quick Start

Create a new post

$ hexo new "My New Post"

More info: Writing

Run server

$ hexo server

More info: Server

Generate static files

$ hexo generate

More info: Generating

Deploy to remote sites

$ hexo deploy

More info: Deployment

测试
$x^{y^z}=(1+{\rm e}^x)^{-2xy^w}$