Skip to main content
Version: WIP

Contribute to Starwhale

Getting Involved/Contributing​

We welcome and encourage all contributions to Starwhale, including and not limited to:

  • Describe the problems encountered during use.
  • Submit feature request.
  • Discuss in Slack and Github Issues.
  • Code Review.
  • Improve docs, tutorials and examples.
  • Fix Bug.
  • Add Test Case.
  • Code readability and code comments to import readability.
  • Develop new features.
  • Write enhancement proposal.

You can get involved, get updates and contact Starwhale developers in the following ways:

Starwhale Resources​

Code Structure​

  • client: swcli and Python SDK with Pure Python3, which includes all Standalone Instance features.
    • api: Python SDK.
    • cli: Command Line Interface entrypoint.
    • base: Python base abstract.
    • core: Starwhale core concepts which includes Dataset,Model,Runtime,Project, job and Evaluation, etc.
    • utils: Python utilities lib.
  • console: frontend with React + TypeScript.
  • server：Starwhale Controller with java, which includes all Starwhale Cloud Instance backend apis.
  • docker：Helm Charts, dockerfile.
  • docs：Starwhale官方文档。
  • example：Example code.
  • scripts：Bash and Python scripts for E2E testing and software releases, etc.

Fork and clone the repository​

You will need to fork the code of Starwhale repository and clone it to your local machine.

  • Fork Starwhale repository: Fork Starwhale Github Repo,For more usage details, please refer to: Fork a repo

  • Install Git-LFS:Git-LFS

     git lfs install

  • Clone code to local machine

    git clone https://github.com/${your username}/starwhale.git

Development environment for Standalone Instance​

Standalone Instance is written in Python3. When you want to modify swcli and sdk, you need to build the development environment.

Standalone development environment prerequisites​

  • OS: Linux or macOS
  • Python: 3.7~3.11
  • Docker: >=19.03(optional)
  • Python isolated env tools：Python venv, virtualenv or conda, etc

Building from source code​

Based on the previous step, clone to the local directory: starwhale, and enter the client subdirectory:

cd starwhale/client

Create an isolated python environment with conda:

conda create -n starwhale-dev python=3.8 -y
conda activate starwhale-dev

Install client package and python dependencies into the starwhale-dev environment:

make install-sw
make install-dev-req

Validate with the swcli --version command. In the development environment, the version is 0.0.0.dev0:

❯ swcli --version
swcli, version 0.0.0.dev0

❯ swcli --version
/home/username/anaconda3/envs/starwhale-dev/bin/swcli

Modifying the code​

When you modify the code, you need not to install python package(run make install-sw command) again. .editorconfig will be imported into the most IDE and code editors which helps maintain consistent coding styles for multiple developers.

Lint and Test​

Run unit test, E2E test, mypy lint, flake lint and isort check in the starwhale directory.

make client-all-check

Development environment for Cloud Instance​

Cloud Instance is written in Java(backend) and React+TypeScript(frontend).

Development environment for Console​

Development environment for Server​

  • Language: Java
  • Build tool: Maven
  • Development framework: Spring Boot+Mybatis
  • Unit test framework：Junit5
    • Mockito used for mocking
    • Hamcrest used for assertion
    • Testcontainers used for providing lightweight, throwaway instances of common databases, Selenium web browsers that can run in a Docker container.
  • Check style tool：use maven-checkstyle-plugin

Server development environment prerequisites​

  • OS: Linux, macOS or Windows
  • Docker: >=19.03
  • JDK: >=11
  • Maven: >=3.8.1
  • Mysql: >=8.0.29
  • Minio
  • Kubernetes cluster/Minikube(If you don't have a k8s cluster, you can use Minikube as an alternative for development and debugging)

Modify the code and add unit tests​

Now you can enter the corresponding module to modify and adjust the code on the server side. The main business code directory is src/main/java, and the unit test directory is src/test/java.

Execute code check and run unit tests​

cd starwhale/server
mvn clean test

Deploy the server at local machine​

  • Dependent services that need to be deployed

    • Minikube（Optional. Minikube can be used when there is no k8s cluster, there is the installation doc: Minikube）

      minikube start
      minikube addons enable ingress
      minikube addons enable ingress-dns

    • Mysql

      docker run --name sw-mysql -d \
      -p 3306:3306 \
      -e MYSQL_ROOT_PASSWORD=starwhale \
      -e MYSQL_USER=starwhale \
      -e MYSQL_PASSWORD=starwhale \
      -e MYSQL_DATABASE=starwhale \
      mysql:latest

    • Minio

      docker run --name minio -d \
      -p 9000:9000  --publish 9001:9001 \
      -e MINIO_DEFAULT_BUCKETS='starwhale' \
      -e MINIO_ROOT_USER="minioadmin" \
      -e MINIO_ROOT_PASSWORD="minioadmin" \
      bitnami/minio:latest

  • Package server program

    If you need to deploy the front-end at the same time when deploying the server, you can execute the build command of the front-end part first, and then execute 'mvn clean package', and the compiled front-end files will be automatically packaged.

    Use the following command to package the program

      cd starwhale/server
      mvn clean package

  • Specify the environment required for server startup

    # Minio env
    export SW_STORAGE_ENDPOINT=http://${Minio IP,default is:27.0.0.1}:9000
    export SW_STORAGE_BUCKET=${Minio bucket,default is:starwhale}
    export SW_STORAGE_ACCESSKEY=${Minio accessKey,default is:starwhale}
    export SW_STORAGE_SECRETKEY=${Minio secretKey,default is:starwhale}
    export SW_STORAGE_REGION=${Minio region,default is:local}
    # kubernetes env
    export KUBECONFIG=${the '.kube' file path}\.kube\config

    export SW_INSTANCE_URI=http://${Server IP}:8082
    export SW_METADATA_STORAGE_IP=${Mysql IP,default: 127.0.0.1}
    export SW_METADATA_STORAGE_PORT=${Mysql port,default: 3306}
    export SW_METADATA_STORAGE_DB=${Mysql dbname,default: starwhale}
    export SW_METADATA_STORAGE_USER=${Mysql user,default: starwhale}
    export SW_METADATA_STORAGE_PASSWORD=${user password,default: starwhale}

  • Deploy server service

    You can use the IDE or the command to deploy.

    java -jar controller/target/starwhale-controller-0.1.0-SNAPSHOT.jar

  • Debug

    there are two ways to debug the modified function:

    • Use swagger-ui for interface debugging, visit /swagger-ui/index.html to find the corresponding api
    • Debug the corresponding function directly in the ui (provided that the front-end code has been built in advance according to the instructions when packaging)