Monday, July 26, 2010

如何在 windows 中创建一个 Cassandra 集群

最近在测试 Cassandra 这种分布式的 NoSql 数据库，配置还是有点麻烦。今天总结下具体的配置过程和注意事项，以下是如何在一台PC上运行一个两个节点的 Cassandra 集群。

配置之前，请确保以下必要条件，

  • JDK/JRE， Cassandra 服务端是个 java 程序，你首先要装一个JDK 
    • 32/64 bit jvm
  • 选择好三四个TCP 端口。
    • Cassandra 存储端口 
      • 7000 是默认端口
    • Thrift 侦听端口
      • 用于跟客户端通讯
      • 9160 默认
    • 两个 JMX 监控的端口,  8080/ 9080
    • JWDP 调试端口（可选）
  • 需要两个 IP 地址

接下来，我们创建一个两个节点名字为  HelloCassendra 的群集。

  • 下载cassandra的压缩文件 http://cassandra.apache.org/ , 我用的是 版本 0.6.3.
    • 解压缩到 本地文件夹，比如 C:\apache-cassandra-0.6.3
  • 复制 Conf 文件夹，两份Copy，分别为Conf1，conf2. 每个文件夹对应一个节点。 
    • C:\apache-cassandra-0.6.3\conf1
    • C:\apache-cassandra-0.6.3\conf2
  • 到文件夹 conf1 下面, 修改节点1的配置(Cassandra 使用一种gossip 集群协议，需要有一些种子节点，我们配置节点1为种子节点)
    • C:\apache-cassandra-0.6.3\conf1\log4j.properties
      • # Edit the next line to point to your logs directory
        log4j.appender.R.File=/var/log/cassandra/system.log
      • 改为 =/var/log/cassandra/c1/system.log
      • 这样会把log写道C1的目录 c:/var/log/cassandra/c1/system.log
    • C:\apache-cassandra-0.6.3\conf1\storage-conf.xml
      • 把群集名字改为 HelloCassandra
        • <ClusterName>Test Cluster</ClusterName>
        • <ClusterName>HelloCassandra</ClusterName>
        • 改一下放置Log和数据的文件夹
        • <CommitLogDirectory>/var/lib/cassandra/C1/commitlog</CommitLogDirectory>
            <DataFileDirectories>
                <DataFileDirectory>/var/lib/cassandra/C1/data</DataFileDirectory>
            </DataFileDirectories>
      • 把Localhost 替换为 127.0.0.1.
        • <ListenAddress>127.0.0.1</ListenAddress>
        • <ThriftAddress>127.0.0.1</ThriftAddress>
  • 到文件夹 conf2 下面, 修改节点2的配置(Cassandra 使用一种gossip 集群协议，需要有一些种子节点，我们配置节点2为普通节点，通过给节点1通讯来建立群集)
    • C:\apache-cassandra-0.6.3\conf2\log4j.properties
      • # Edit the next line to point to your logs directory
        log4j.appender.R.File=/var/log/cassandra/system.log
      •  =/var/log/cassandra/c2/system.log
      • 这样节点log写到C2目录 c:/var/log/cassandra/c2/system.log
    • C:\apache-cassandra-0.6.3\conf2\storage-conf.xml
      • 群集名字修改为HelloCassandra
        • <ClusterName>Test Cluster</ClusterName>
        • <ClusterName>HelloCassandra</ClusterName>
      • 修改 commitlogdirectory and DataFileDirectory 文件夹 
        • <CommitLogDirectory>/var/lib/cassandra/C2/commitlog</CommitLogDirectory>
            <DataFileDirectories>
                <DataFileDirectory>/var/lib/cassandra/C3/data</DataFileDirectory>
            </DataFileDirectories>
      • 把localhost 替换为 127.0.0.2 ，每个节点需要不同的IP地址。节点2 使用127.0.0.2.
        • <ListenAddress>127.0.0.2</ListenAddress>
        • <ThriftAddress>127.0.0.2</ThriftAddress>
        • 启动AutoBootTrap模式，自动从其他节点分配数据
        • <AutoBootstrap>false</AutoBootstrap>
  • 到文件 C:\apache-cassandra-0.6.3\bin 下面, 复制 cassandra.bat ，另存为 c1.bat， c2.bat.每一个批处理文件对应一个实例的启动脚本。
    • C:\apache-cassandra-0.6.3\bin\c1.bat
    • C:\apache-cassandra-0.6.3\bin\c2.bat
  • 编辑 c1.bat,指向Conf1目录下面的配置，修改JMX 端口以及调试端口。这里我们禁用调试
    • if NOT DEFINED CASSANDRA_CONF set CASSANDRA_CONF=%CASSANDRA_HOME%\conf
    • if NOT DEFINED CASSANDRA_CONF set CASSANDRA_CONF=%CASSANDRA_HOME%\conf1
    • 删除调试这一行
    • -Xrunjdwp:transport=dt_socket,server=y,address=8888,suspend=n^
    • 对于实例1，保持默认端口8080
    • -Dcom.sun.management.jmxremote.port=8080^
  • 编辑 c2.bat,指向Conf2目录下面的配置，修改JMX 端口以及调试端口。这里我们禁用调试 
    • if NOT DEFINED CASSANDRA_CONF set CASSANDRA_CONF=%CASSANDRA_HOME%\conf
      • if NOT DEFINED CASSANDRA_CONF set CASSANDRA_CONF=%CASSANDRA_HOME%\conf2
    • 删除调试这一行
      • -Xrunjdwp:transport=dt_socket,server=y,address=8888,suspend=n^
    • 对于实例2，替换默认端口为9080
      • -Dcom.sun.management.jmxremote.port=9080^
  • 准备启动两个批处理。

启动C1.bat，你可以看到他是个Seed 节点。

Starting Cassandra Server
INFO 16:04:31,597 Auto DiskAccessMode determined to be mmap
INFO 16:04:31,909 Saved Token not found. Using 22656600690150525193669162742751150004
INFO 16:04:31,909 Saved ClusterName not found. Using HelloCassandra
INFO 16:04:31,909 Creating new commitlog segment /var/lib/cassandra/c1/commitlog\CommitLog-1280185471909.log
INFO 16:04:31,987 LocationInfo has reached its threshold; switching in a fresh Memtable at CommitLogContext(file='/var/lib/cassandra/c1/commitlog\C
INFO 16:04:31,987 Enqueuing flush of Memtable-LocationInfo@1351579886(171 bytes, 4 operations)
INFO 16:04:31,987 Writing Memtable-LocationInfo@1351579886(171 bytes, 4 operations)
INFO 16:04:32,236 Completed flushing C:\var\lib\cassandra\c1\data\system\LocationInfo-1-Data.db
INFO 16:04:32,283 Starting up server gossip
INFO 16:04:32,299 This node will not auto bootstrap because it is configured to be a seed node.
INFO 16:04:32,346 Binding thrift service to /127.0.0.1:9160
INFO 16:04:32,346 Cassandra starting up...


启动好之后，你可以看到这个结点侦听的端口
image

这里7000是存储端口，8080是jmx端口，9160 是Thrift 端口。   
  
这时候cluster中只有一个节点

C:\apache-cassandra-0.6.3>bin\nodetool --host 127.0.0.1 --port 8080 ring
Starting NodeTool
Address       Status     Load          Range                                      Ring
127.0.0.1     Up         497 bytes     22656600690150525193669162742751150004     |<--|

然后启动c2.bat，这里可能要等120秒。90秒load配置，30秒来分配数据
 

Starting Cassandra Server
INFO 16:11:58,940 Auto DiskAccessMode determined to be mmap
INFO 16:11:59,237 Saved Token not found. Using 168810650452358861593947197964955051846
INFO 16:11:59,252 Saved ClusterName not found. Using HelloCassandra
INFO 16:11:59,252 Creating new commitlog segment /var/lib/cassandra/c2/commitlog\CommitLog-1280185919252.log
INFO 16:11:59,315 LocationInfo has reached its threshold; switching in a fresh Memtable at CommitLogContext(file='/var/lib/cassandra/c2/commitlog\Com
INFO 16:11:59,315 Enqueuing flush of Memtable-LocationInfo@625647261(171 bytes, 4 operations)
INFO 16:11:59,315 Writing Memtable-LocationInfo@625647261(171 bytes, 4 operations)
INFO 16:11:59,564 Completed flushing C:\var\lib\cassandra\c2\data\system\LocationInfo-1-Data.db
INFO 16:11:59,596 Starting up server gossip
INFO 16:11:59,627 Joining: getting load information
INFO 16:11:59,627 Sleeping 90000 ms to wait for load information...
INFO 16:12:01,577 Node /127.0.0.1 is now part of the cluster
INFO 16:12:02,592 InetAddress /127.0.0.1 is now UP
INFO 16:12:02,592 Started hinted handoff for endPoint /127.0.0.1
INFO 16:12:02,607 Finished hinted handoff of 0 rows to endpoint /127.0.0.1
INFO 16:13:29,657 Joining: getting bootstrap token
INFO 16:16:44,916 New token will be 107727192420385141059512814600693202868 to assume load from /127.0.0.1
INFO 16:16:44,931 Joining: sleeping 30000 ms for pending range setup
INFO 16:17:14,952 Bootstrapping
INFO 16:17:15,030 Bootstrap/move completed! Now serving reads.
INFO 16:17:15,108 Binding thrift service to /127.0.0.2:9160
INFO 16:17:15,108 Cassandra starting up...

在看tcpview，你可以看到两个ip通过存储端口建立会话
image 
时候就可以看到群集有两个成员了。

C:\apache-cassandra-0.6.3>bin\nodetool --host 127.0.0.1 --port 8080 ring
Starting NodeTool
Address       Status     Load          Range                                      Ring
                                       107727192420385141059512814600693202868
127.0.0.1     Up         497 bytes     22656600690150525193669162742751150004     |<--|
127.0.0.2     Up         497 bytes     107727192420385141059512814600693202868    |-->|

 

你也可以用jconsole，端口8080/9080 来查看更细节的信息。如下图

image

 

Now, everything is set. enjoy you exploring.

Posted by Ryan at 9:15 PM
Labels:

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)