Author: van

constructor

    <resultMap id="UserBasic" type="com.alibaba.force.dal.model.capsulation.UserBasic">
        <constructor>
            <arg column="u_name" jdbcType="VARCHAR" javaType="java.lang.String"/>
            <arg column="u_username" jdbcType="VARCHAR" javaType="java.lang.String"/>
            <arg column="u_id" jdbcType="BIGINT" javaType="java.lang.Long"/>
            <arg column="u_state" jdbcType="VARCHAR" javaType="java.lang.String"/>
            <arg column="u_avatar" jdbcType="VARCHAR" javaType="java.lang.String"/>
            <arg column="u_extern_uid" jdbcType="VARCHAR" javaType="java.lang.String"/>
        </constructor>
    </resultMap>

@ToString
@Getter
@Setter
public class UserBasic extends UserSafe {
    private Long id;

    private String state;

    @JSONField(name = "avatar_url")
    private String avatarUrl;

    @JSONField(name = "extern_uid")
    private String externUid;

    /**
     * @param name
     * @param username
     * @param id
     * @param state
     * @param avatar    注意，这里需要做一些额外操作
     * @param externUid
     */
    public UserBasic(String name, String username, Long id, String state, String avatar, String externUid) {
        super(name, username);
        this.id = id;
        this.state = state;
        this.externUid = externUid;
        this.setAvatarUrl(UsersUtil.getAvatarUrl(UsersBuilder.newBuilder().setId(id).setAvatar(avatar)
            .setExternUid(externUid).build()));
    }

    public UserBasic() {
    }

    @Override
    public UserBasic merge(@NonNull Users users) {
        super.merge(users);
        this.setId(users.getId());
        this.setState(users.getState());
        this.setAvatarUrl(UsersUtil.getAvatarUrl(users));
        this.setExternUid(users.getExternUid());
        return this;
    }
}

注意入参的顺序保持一致

association

association字段可以直接增加ResultMap的子属性。

    <resultMap id="MigrateTaskRecordDO" type="com.alibaba.force.dal.model.capsulation.MigrateTaskRecordDO">
  <id column="id" jdbcType="BIGINT" property="id" />
  <id column="u_id"/>
  <result column="gmt_create" jdbcType="TIMESTAMP" property="gmtCreate" />
  <result column="gmt_modified" jdbcType="TIMESTAMP" property="gmtModified" />
  <association property="creator" javaType**="com.alibaba.force.dal.model.capsulation.UserBasic"    resultMap="UserBasic"/>
</resultMap>

association中的resultMap所指向的字段也必须是申明过的ResultMap对象。
result需要放置在association之前。

N+1问题

注意上面例子中的多个标签

有这么一个场景，会产生坑：
当查询1对多的关系时，如果采用了嵌套的方式（association、collection），则MyBatis会在DefaultResultSetHandler中调用handleRowValuesForNestedResultMap
方法。
该方法采用了计算key(createRowKey)的方式来记录已经遍历过的对象，并且下次从缓存中拿（减少内存的消耗，共享嵌套的重复对象）。

而由于N+1的select一定是根据这个“1”去查询其他的“N”的，如果此时<id>不变，为主键，那么就会被丢弃很多结果（理论上不重复，但是由于上述的计算规则，被MyBatis认为是重复）。因此，使用多个联合字段作为id是最棒的解。这些联合ID需要是最终确定一个不重复对象的唯一值。

MyBatis版本：3.4.4

In Spring, we can use FilterChainProxy to manage custom filters properly.

Custom resource xml file

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">

    <bean id="webFilterChain" class="org.springframework.security.web.DefaultSecurityFilterChain">
        <constructor-arg name="requestMatcher">
            <bean class="org.springframework.security.web.util.matcher.NegatedRequestMatcher">
                <constructor-arg name="requestMatcher">
                    <bean class="org.springframework.security.web.util.matcher.RegexRequestMatcher">
                        <constructor-arg name="httpMethod" value=""/>
                        <constructor-arg name="pattern" value="/api/.*"/>
                    </bean>
                </constructor-arg>
            </bean>
        </constructor-arg>
        <constructor-arg name="filters">
            <list>
                <bean class="org.springframework.security.web.csrf.CsrfFilter">
                    <constructor-arg name="csrfTokenRepository">
                        <bean class="org.springframework.security.web.csrf.CookieCsrfTokenRepository"/>
                    </constructor-arg>
                </bean>
                <ref bean="springSessionRepositoryFilter"/>
            </list>
        </constructor-arg>
    </bean>

    <bean id="customFilterChainProxy" class="org.springframework.security.web.FilterChainProxy">
        <constructor-arg>
            <list>
                <ref bean="webFilterChain"/>
            </list>
        </constructor-arg>
    </bean>

</beans>

web.xml filter configuration

    <context-param>
        <param-name>contextConfigLocation</param-name>
        <param-value>
            <!-- ... -->
            classpath:spring/filter.xml
        </param-value>
    </context-param>

 <!-- Custom Filter -->
    <filter>
        <filter-name>customFilterChainProxy</filter-name>
        <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>
    </filter>
    <filter-mapping>
        <filter-name>customFilterChainProxy</filter-name>
        <url-pattern>/*</url-pattern>
    </filter-mapping>

Explain

customFilterChainProxy filter-name in web.xml. This tells Container(Tomcat) to add filter by Class org.springframework.web.filter.DelegatingFilterProxy.

DelegatingFilterProxy : Spring will search filter-name (customFilterChainProxy) in Spring context for Good Management.And customFilterChainProxy is initialized in filter.xml resource file.

SpringFox

package com.alibaba.force.api.shared.swagger;

import java.util.ArrayList;
import java.util.List;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.servlet.config.annotation.EnableWebMvc;
import springfox.documentation.builders.ParameterBuilder;
import springfox.documentation.builders.PathSelectors;
import springfox.documentation.builders.RequestHandlerSelectors;
import springfox.documentation.schema.ModelRef;
import springfox.documentation.service.ApiInfo;
import springfox.documentation.service.Contact;
import springfox.documentation.service.Parameter;
import springfox.documentation.service.Tag;
import springfox.documentation.spi.DocumentationType;
import springfox.documentation.spring.web.plugins.Docket;
import springfox.documentation.swagger2.annotations.EnableSwagger2;

/**
 * @author van.yzt
 * @date 2017/07/31
 */
@Configuration
@EnableSwagger2
@EnableWebMvc
public class SwaggerConfig {
    @Bean
    public Docket healthDocket() {
        return new Docket(DocumentationType.SWAGGER_2).groupName("Health").select()
            .apis(RequestHandlerSelectors.basePackage("com.alibaba.force.api.health"))
            .paths(PathSelectors.any()).build().tags(new Tag("System API", "This is for health."))
            .useDefaultResponseMessages(false)
            .apiInfo(apiInfo("System API", "This is Force system API"));
    }

    @Bean
    public Docket internalDocket() {
        return new Docket(DocumentationType.SWAGGER_2).groupName("Internal").select()
            .apis(RequestHandlerSelectors.basePackage("com.alibaba.force.api.internal"))
            .paths(PathSelectors.any()).build().tags(new Tag("System API", "This is for internal API."))
            .useDefaultResponseMessages(false)
            .globalOperationParameters(privateToken())
            .apiInfo(apiInfo("Internal API", "This is Force internal API"));
    }

    @Bean
    public Docket repositoryDocket() {
        return new Docket(DocumentationType.SWAGGER_2).groupName("Repository").select()
            .apis(RequestHandlerSelectors.basePackage("com.alibaba.force.api.repository"))
            .paths(PathSelectors.any()).build().tags(new Tag("Repository API", "This is for repository api."))
            .useDefaultResponseMessages(false)
            .globalOperationParameters(privateToken())
            .apiInfo(apiInfo("Repository API", "This is Force repository API"));
    }

    @Bean
    public Docket userDocket() {
        return new Docket(DocumentationType.SWAGGER_2).groupName("User").select()
            .apis(RequestHandlerSelectors.basePackage("com.alibaba.force.api.user"))
            .paths(PathSelectors.any()).build().tags(new Tag("User API", "This is for user api."))
            .useDefaultResponseMessages(false)
            .globalOperationParameters(privateToken())
            .apiInfo(apiInfo("User API", "This is Force user API"));
    }

    @Bean
    public Docket keyDocket() {
        return new Docket(DocumentationType.SWAGGER_2).groupName("Key").select()
            .apis(RequestHandlerSelectors.basePackage("com.alibaba.force.api.key"))
            .paths(PathSelectors.any()).build().tags(new Tag("Key API", "This is for key api."))
            .useDefaultResponseMessages(false)
            .globalOperationParameters(privateToken())
            .apiInfo(apiInfo("Key API", "This is Force key API"));
    }

    private List<Parameter> privateToken() {
        return new ArrayList<Parameter>() {{
            add(new ParameterBuilder()
                .name("private_token")
                .description("Private token for person")
                .modelRef(new ModelRef("String"))
                .parameterType("query")
                .required(true)
                .build());
        }};
    }

    private ApiInfo apiInfo(String title, String description) {
        return new ApiInfo(title, description, "v3", "", DEFAULT_CONTACT, null, null);
    }

    private static final Contact DEFAULT_CONTACT = new Contact("van.yzt",
        "https://lark.alipay.com/aone/code+voice/gt7niw", "van.yzt@alibaba-inc.com");
}

Register custom converter:

When use Spring <mvc:annotation-driven/>, Spring will automatically initialize some MVC usefull bean @See WebMvcConfigurerComposite. It contains :

@Bean
public FormattingConversionService mvcConversionService() {
    FormattingConversionService conversionService = new DefaultFormattingConversionService();
    addFormatters(conversionService);
    return conversionService;
}

And then, <mvc:annotation-driven conversion-service="mvcConversionService" /> and we can you it:

/**
 * @author van.yzt
 * @date 2017/08/29
 */
@Configuration
@EnableWebMvc
public class WebConfig extends WebMvcConfigurerAdapter {

    @Override
    public void addFormatters(FormatterRegistry registry) {
        registry.addConverter(new StringToAccessLevelEnum());
        super.addFormatters(registry);
    }

    final class StringToAccessLevelEnum implements Converter<String, AccessLevelEnum> {
        @Override
        public AccessLevelEnum convert(String source) {
            return AccessLevelEnum.fromValue(Integer.parseInt(source));
        }
    }
}

pom.xml

<dependency>
    <groupId>org.springframework.session</groupId>
    <artifactId>spring-session-data-redis</artifactId>
    <version>1.3.1.RELEASE</version>
    <type>pom</type>
    <exclusions>
        <exclusion>
            <groupId>org.springframework.data</groupId>
            <artifactId>spring-data-redis</artifactId>
        </exclusion>
    </exclusions>
</dependency>

...

<dependency>
    <groupId>org.springframework.data</groupId>
    <artifactId>spring-data-redis</artifactId>
    <version>1.8.6.RELEASE</version>
</dependency>

WebConfig.java

package com.alibaba.force.web.config;

import com.alibaba.force.api.shared.aop.ForceHandlerInterceptor;
import com.alibaba.force.common.enums.VisibilityLevelEnum;
import com.alibaba.force.common.util.PropertyUtils;
import com.alibaba.force.dal.model.AccessLevelEnum;

import com.lambdaworks.redis.resource.ClientResources;
import com.lambdaworks.redis.resource.DefaultClientResources;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.convert.converter.Converter;
import org.springframework.data.redis.connection.RedisSentinelConfiguration;
import org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory;
import org.springframework.format.FormatterRegistry;
import org.springframework.session.data.redis.config.annotation.web.http.EnableRedisHttpSession;
import org.springframework.web.servlet.config.annotation.EnableWebMvc;
import org.springframework.web.servlet.config.annotation.InterceptorRegistry;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurerAdapter;

/** 
  * @author van.yzt
  * @date 2017/08/29
  */
  @Configuration
  @EnableWebMvc
  @EnableRedisHttpSession(maxInactiveIntervalInSeconds = 108000)
  public class WebConfig extends WebMvcConfigurerAdapter {
  ...
   @Bean
   public LettuceConnectionFactory connectionFactory() {
       RedisSentinelConfiguration redisSentinelConfiguration = new RedisSentinelConfiguration(
           PropertyUtils.getRedisSentinelMasterName(),
           PropertyUtils.getRedisSentinelNodes());
       LettuceConnectionFactory lettuceConnectionFactory = new LettuceConnectionFactory(redisSentinelConfiguration);
       ClientResources clientResources = new LettuceRedisClientResourceConfig(DefaultClientResources.builder());
       lettuceConnectionFactory.setClientResources(clientResources);
       lettuceConnectionFactory.setTimeout(5000);
       lettuceConnectionFactory.setPassword(PropertyUtils.getRedisConfigPassword());
       lettuceConnectionFactory.setShareNativeConnection(true);
       return lettuceConnectionFactory;
   }
  ...
  }

This configuration uses Redis Sentinel.

Configuration

redis.sentinel.master.name=force-master
redis.sentinel.nodes=10.101.104.13:26379,10.101.104.13:26380,10.101.104.13:26381
redis.config.password=YPr2Ymj^8$8a2*aD0F7z^L

采用In-Process作为Transport协议，直接生成一个本地的Server，将Server端mock，从而实现Clinet端的代码测试。gRPC的维护者们不允许Client去做mock。

1. 实现本地In-Process的Server端；
   1. 实现由Protocol Buffer生成的ServiceBase对象，覆盖它的具体方法；（该类的作用就是server执行的具体实现）
   2. 将上述实现注册到本地In-Process Server；（之后Client调用Server就能由我们自己实现的ServiceBase来完成数据的mock，官方生成的类里，实现是抛错……）
2. 获取In-Process Server中的Channel，以提供Client生成Stub；
3. 跑测试；

依照上述步骤，结合原力项目（代码服务）场景：
我们需要对某一个路径下的仓库检测其是否存，用gRPC协议调用远端的Server完成。

Projects.java(业务Service)

   /** 
     *{@inheritDoc}
     *
     * @param projectPathWithNamespace
     * @return
     */
    @Override
    public Optional<Projects> isExist(@NonNull String projectPathWithNamespace) {
        Optional<Projects> projectsOptional = projectsDao.findByNamespace(projectPathWithNamespace);
        if (projectsOptional.isPresent()) {
            RepositoryExistsRequest repositoryExistsRequest = RepositoryExistsRequest.newBuilder()
                .setRepository(GrpcUtil.generateRepository(projectPathWithNamespace)).build();
            RepositoryServiceBlockingStub stub = grpcStub.get(RepositoryServiceBlockingStub.class,
                GrpcUtil.getNamespace(projectPathWithNamespace));
            RepositoryExistsResponse response = stub.repositoryExists(repositoryExistsRequest);
            // todo 通知清理脏数据
            if (!response.getExists()) {
                return Optional.empty();
            }
        }
        return projectsOptional;
    }

系统在这里统一处理了stub的生成：grpcStub.get(...)

扩展RepositoryServiceGrpc下的RepositoryServiceImplBase（Protocol Buffer自动生成Service）

/**
 * Extend {@link RepositoryServiceGrpc.RepositoryServiceImplBase} in order to implement the mock method.
 *
 * @author van.yzt
 * @date 2017/09/27
 */
public class MockRepositoryServiceImplBase extends RepositoryServiceGrpc.RepositoryServiceImplBase {

    /**
     * Implement the exist check method.
     *
     * @param request
     * @param responseObserver
     */
    @Override
    public void repositoryExists(RepositoryExistsRequest request,
        StreamObserver<RepositoryExistsResponse> responseObserver) {
        responseObserver.onNext(RepositoryExistsResponse.newBuilder().setExists(true).build());
        responseObserver.onCompleted();
    }
}

ProjectsTest.java

@Rule
private GrpcStub grpcStub = new MockGrpcStub();
...

@Before

public void setUp() throws Exception {

    RepositoryServiceImplBase repositoryServiceImplBase = Mockito.spy(new MockRepositoryServiceImplBase() {});
    // 注册到In-Process中

    grpcStub.getServiceRegistry().addService(repositoryServiceImplBase);

}


 @Test
    public void isExistTest() throws Exception {
        Mockito.when(mockProjectsDao.findByNamespace(TestDataBuilder.ABSOLUTE_PATH)).thenReturn(
            Optional.of(TestDataBuilder.buildProjects()));
        Assert.assertNotNull(projects.isExist(TestDataBuilder.ABSOLUTE_PATH));
    }

MockGrpcStub.java（业务相关的封装，用于生成Stub的Service）

/**
 * This stub generate service is used for unit test.
 *
 * @author van.yzt
 * @date 2017/09/27
 */
public class MockGrpcStub extends ExternalResource implements AbstractMockGrpcStub {
    private ManagedChannel channel;
    private Server server;
    private String serverName;
    private MutableHandlerRegistry serviceRegistry;
    private boolean useDirectExecutor;

    public AbstractMockGrpcStub directExecutor() {
        useDirectExecutor = true;
        return this;
    }


    /**
     * 获取固定的headers
     *
     * @param namespace
     * @param timestamp
     * @return
     */
    private Map<String, String> headers(@NonNull String namespace, @NonNull Long timestamp) {
        final Map<String, String> headers = new HashMap<>(2);
        // Some operations
        return headers;
    }

    @SuppressWarnings("unchecked")
    @Override
    public <T extends AbstractStub<T>> T get(@NonNull Class<T> stubClass, @NonNull final String namespace) {
        try {
            Constructor constructor = stubClass.getDeclaredConstructor(Channel.class);
            constructor.setAccessible(true);
            AbstractStub stub = (AbstractStub)constructor.newInstance(channel);
            /**
             * 不需要关心并发
             */
            stub = attachHeaders(stub, headers(namespace, System.currentTimeMillis()));
            return (T)stub;
        } catch (InvocationTargetException | InstantiationException | IllegalAccessException e) {
            throw new GrpcStubCreateException("gRPC create stub failed.", e);
        } catch (NoSuchMethodException e) {
            /**
             * This place means Google Protocol Buffers generate result format changed !!!!!
             *
             * When writing this code, the XXXBlockingStub class has private constructor like
             * 'XXXBlockingStub(io.grpc.Channel)'
             */
            throw new GrpcStubCreateException("gRPC create stub failed because of the format changed.", e);
        }
    }

    /**
     * Returns a {@link ManagedChannel} connected to this service.
     */
    public final ManagedChannel getChannel() {
        return channel;
    }

    /**
     * Returns the underlying gRPC {@link Server} for this service.
     */
    public final Server getServer() {
        return server;
    }

    /**
     * Returns the randomly generated server name for this service.
     */
    public final String getServerName() {
        return serverName;
    }

    /**
     * Returns the service registry for this service. The registry is used to add service instances
     * (e.g. {@link io.grpc.BindableService} or {@link io.grpc.ServerServiceDefinition} to the server.
     */
    public final MutableHandlerRegistry getServiceRegistry() {
        return serviceRegistry;
    }

    /**
     * After the test has completed, clean up the channel and server.
     */
    @Override
    protected void after() {
        serverName = null;
        serviceRegistry = null;

        channel.shutdown();
        server.shutdown();

        try {
            channel.awaitTermination(1, TimeUnit.MINUTES);
            server.awaitTermination(1, TimeUnit.MINUTES);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
            throw new RuntimeException(e);
        } finally {
            channel.shutdownNow();
            channel = null;

            server.shutdownNow();
            server = null;
        }
    }

    /**
     * Before the test has started, create the server and channel.
     */
    @Override
    protected void before() throws Throwable {
        serverName = UUID.randomUUID().toString();

        serviceRegistry = new MutableHandlerRegistry();

        InProcessServerBuilder serverBuilder = InProcessServerBuilder.forName(serverName)
            .fallbackHandlerRegistry(serviceRegistry);

        if (useDirectExecutor) {
            serverBuilder.directExecutor();
        }

        server = serverBuilder.build().start();

        InProcessChannelBuilder channelBuilder = InProcessChannelBuilder.forName(serverName);

        if (useDirectExecutor) {
            channelBuilder.directExecutor();
        }

        channel = channelBuilder.build();
    }

Mockito.when(mockTransactionTemplate.execute(Mockito.any()))
    .thenAnswer(invocationOnMock -> {
        Object[] args = invocationOnMock.getArguments();
        TransactionCallback arg = (TransactionCallback)args[0];
        return arg.doInTransaction(new SimpleTransactionStatus());
    });

Delete the pattern line and save back (Mac OSX)\nMac use sed command’s -i option need String parameter , in order to tell sed save old file to this file suffix.

find . -type f -name '*[m|h]' | xargs -I {} sed -i '' '/.*凌星.*/d' {}

#!/usr/bin/expect -f

proc terminal:password:get {promptString} {

     # Turn off echoing, but leave newlines on.  That looks better.
     # Note that the terminal is left in cooked mode, so people can still use backspace
     exec stty -echo echonl <@stdin

     # Print the prompt
     puts -nonewline stdout $promptString
     flush stdout

     # Read that password!  :^)
     gets stdin password

     # Reset the terminal
     exec stty echo -echonl <@stdin

     return $password
}

proc chooseMachine { } {
    puts "1) 11.192.101.216 (pre/ccvob);\n2) 11.192.101.192;\n3) 11.192.101.236;\n4) 11.192.101.221;";
    gets stdin choice;
    switch $choice {
        1 {
            return "11.192.101.216";
        }
        2 {
            return "11.192.101.192";
        }
        3 {
            return "11.192.101.236";
        }
        4 {
            return "11.192.101.221";
        }
        default {
            return "11.192.101.216";
        }
    }
}

proc main { } {
    set timeout -1;
    set IP [ chooseMachine ];
    set PASSWORD [ terminal:password:get "请输入密码:" ];
    puts "Start build source success";
    spawn ssh van.yzt@11.239.182.250;
    expect "*password:";
    send "$PASSWORD\r";
    expect "*$*";
    set TOKEN [ terminal:password:get "请输入6位令牌:" ];
    spawn ssh van.yzt@login1.am100.alibaba-inc.com;
    expect "*password*";
    send "$PASSWORD$TOKEN\r";
    expect "*$*";
    send "ssh van.yzt@$IP\r";
    expect "password:";
    send "$PASSWORD\r";
    expect "*$*";
    send "sudo su admin\r";
    expect "*password*";
    send "$PASSWORD\r";
    expect "*$*";
    interact;
    exit 0;
}

[ main ];