Convert Figma logo to code with AI

Leon406 logoSubCrawler

节点爬取,筛选, google ping,支持Clash,base64订阅解析,生成可用的ss, ssr, v2ray, trojan,vless节点. (若不进行二次开发,请不要fork)

1,187
203
1,187
0

Top Related Projects

Utility to convert between various subscription format

3,266

SSR 去广告ACL规则/SS完整GFWList规则/Clash规则碎片,Telegram频道订阅地址

5,003

机场推荐/SSR V2ray节点订阅机场/镜像直连/工具推荐

Quick Overview

SubCrawler is a tool for collecting and aggregating subscription links from various sources. It's designed to help users gather and manage subscription information for services like VPN or proxy servers. The project is primarily written in Kotlin and includes features for link parsing, validation, and output generation.

Pros

  • Supports multiple subscription link formats and sources
  • Includes built-in link validation and deduplication
  • Offers various output formats for collected links
  • Regularly updated with new features and improvements

Cons

  • Limited documentation, which may make it challenging for new users
  • Primarily focused on Chinese language sources and services
  • May require additional configuration for use with specific services
  • Potential legal and ethical concerns regarding the collection and sharing of subscription links

Code Examples

// Example 1: Parsing a subscription link
val parser = SubscriptionParser()
val subscriptionInfo = parser.parse("https://example.com/subscription")
println(subscriptionInfo)
// Example 2: Validating a list of links
val validator = LinkValidator()
val validLinks = validator.validateLinks(listOfLinks)
validLinks.forEach { println(it) }
// Example 3: Generating output in a specific format
val generator = OutputGenerator()
val jsonOutput = generator.generateJson(validLinks)
println(jsonOutput)

Getting Started

To use SubCrawler in your project:

  1. Clone the repository:

    git clone https://github.com/Leon406/SubCrawler.git
    
  2. Build the project using Gradle:

    ./gradlew build
    
  3. Run the main application:

    java -jar build/libs/SubCrawler.jar
    
  4. Configure the sources and output options in the config.properties file.

  5. Use the generated output files or integrate the library into your own Kotlin/Java project.

Competitor Comparisons

Utility to convert between various subscription format

Pros of subconverter

  • More comprehensive feature set for subscription conversion and management
  • Supports a wider range of proxy protocols and configuration formats
  • Better documentation and user guides

Cons of subconverter

  • More complex setup and configuration process
  • Heavier resource usage due to its extensive features
  • Steeper learning curve for new users

Code Comparison

SubCrawler (Kotlin):

fun parseClash(text: String): List<Sub> {
    return parseYaml(text).proxies?.map { proxy ->
        Sub(
            type = proxy.type,
            name = proxy.name,
            server = proxy.server,
            port = proxy.port
        )
    } ?: emptyList()
}

subconverter (C++):

void rulesetToClash(YAML::Node &base_rule, std::string &rule_name, std::string &custom_port)
{
    base_rule["name"] = rule_name;
    base_rule["type"] = "http";
    base_rule["behavior"] = "classical";
    base_rule["path"] = "./ruleset/" + rule_name + ".yaml";
    if(!custom_port.empty())
        base_rule["url"] = "http://www.gstatic.com/generate_204";
}

The code snippets show different approaches to handling configuration data, with SubCrawler focusing on parsing and subconverter on generating rule sets.

3,266

SSR 去广告ACL规则/SS完整GFWList规则/Clash规则碎片,Telegram频道订阅地址

Pros of ACL4SSR

  • More comprehensive rule sets for various applications and services
  • Regular updates and maintenance by an active community
  • Includes pre-configured lists for popular platforms like Clash and Quantumult X

Cons of ACL4SSR

  • Primarily focused on Chinese websites and services
  • May require more manual configuration for non-Chinese users
  • Less flexibility in customizing individual rules compared to SubCrawler

Code Comparison

SubCrawler:

fun crawlSubscription(url: String): List<String> {
    val response = client.newCall(Request.Builder().url(url).build()).execute()
    return response.body?.string()?.split("\n") ?: emptyList()
}

ACL4SSR:

payload:
  - DOMAIN-SUFFIX,googleapis.cn
  - DOMAIN-SUFFIX,googleapis.com
  - DOMAIN-SUFFIX,google.cn
  - DOMAIN-SUFFIX,google.com

SubCrawler focuses on crawling and processing subscription URLs, while ACL4SSR provides pre-defined rule sets in YAML format. SubCrawler offers more flexibility for custom crawling and processing, whereas ACL4SSR provides ready-to-use rules for popular services and platforms.

5,003

Pros of sub-web

  • User-friendly web interface for subscription conversion and management
  • Supports multiple subscription formats and protocols
  • Easy deployment with Docker

Cons of sub-web

  • Limited crawling capabilities compared to SubCrawler
  • Less frequent updates and maintenance
  • Fewer advanced features for proxy testing and filtering

Code Comparison

sub-web (Vue.js component):

<template>
  <div class="subscription-card">
    <el-input v-model="url" placeholder="Enter subscription URL"></el-input>
    <el-button @click="convertSubscription">Convert</el-button>
  </div>
</template>

SubCrawler (Kotlin code):

fun crawlSubscriptions(urls: List<String>): List<Proxy> {
    return urls.flatMap { url ->
        val response = httpClient.get(url)
        parseProxies(response.body)
    }
}

Summary

SubCrawler focuses on crawling and aggregating proxy subscriptions, offering more advanced features for proxy testing and filtering. It's primarily a command-line tool written in Kotlin.

sub-web provides a web-based interface for subscription conversion and management, making it more accessible for users who prefer a graphical interface. It's built with Vue.js and supports various subscription formats.

Both projects serve different use cases within the proxy management ecosystem, with SubCrawler being more suitable for advanced users and automation, while sub-web caters to those seeking a user-friendly web interface for subscription handling.

机场推荐/SSR V2ray节点订阅机场/镜像直连/工具推荐

Pros of Share-SSR-V2ray

  • Provides a curated list of SSR and V2Ray servers, making it easier for users to find reliable connections
  • Includes detailed instructions and tutorials for setting up various VPN clients
  • Regularly updated with new server information and configuration guides

Cons of Share-SSR-V2ray

  • Less automated than SubCrawler, requiring manual updates and maintenance
  • Limited to specific VPN protocols (SSR and V2Ray), while SubCrawler supports a wider range
  • Lacks built-in tools for testing or validating server connections

Code Comparison

Share-SSR-V2ray (README.md):

## SSR 订阅链接
- https://raw.githubusercontent.com/ssrsub/ssr/master/ssrsub
- https://www.liesauer.net/yogurt/subscribe?ACCESS_TOKEN=DAYxR3mMaZAsaqUb

SubCrawler (build.gradle.kts):

dependencies {
    implementation("org.jsoup:jsoup:1.14.3")
    implementation("com.squareup.okhttp3:okhttp:4.9.3")
    implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.0")
}

The code comparison shows that Share-SSR-V2ray primarily consists of markdown files with server information, while SubCrawler is a Kotlin-based project with dependencies for web scraping and asynchronous programming.

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

访客数 :eyes:

Sub :: Visitor's Count Leon406:: Visitor's Count

Repo size

Telegram群组

优质节点池

大部分都停用,自行搜索

节点池搭建

有服务器的可自行搭建 proxypool 配置文件 source.yaml

订阅转换

节点测速

  • 本地测速

  • 在线测速(基于上面本地测速搭建的服务,建议自行搭建,目前大部分都挂了)

  • 国内自有服务器Linux测速,使用litespeed

    # litespeed测速,建议关闭网速测试
    ./lite --config config.json --test 生成链接
    # 复制到静态nginx资源路径
    cp output.txt  /usr/share/nginx/res/nodes.txt
    # base64编码,生成v2ray订阅
    base64  /usr/share/nginx/res/nodes.txt > /usr/share/nginx/res/node.txt
    
    

    参考config.json 配置

    
    {
      "group": "Default",
      "speedtestMode": "pingonly",
      "pingMethod": "googleping",
      "sortMethod": "rspeed",
      "concurrency": 1024,
      "testMode": 2,
      "timeout": 5,
      "fontSize": 24,
      "outputMode": 4,
      "unique": true,
      "language": "en",
      "theme": "rainbow"
    }
    

节点过滤

删除以下可能存在测速问题的节点

  • SSR
    • none
    • rc4
    • rc4-md5
  • SS
    • aes-128-cfb
    • aes-256-cfb
    • rc4-md5
  • VMESS
    • none
    • grpc
    • h2
    • auto

项目生成内容

节点

## windows系统执行
localFilter.bat
## Linux /Mac OS系统执行
bash localFilter  
## 或者
chmod +x localFilter && ./localFilter

默认生成的为base64编码(v2rayN/ss/ssr等客户端可直接使用),其他请自行使用订阅转换进行转换

Hosts

走代理后ip匿名检测

使用软件

平台软件支持协议
WindowsV2rayNSS、Trojan、Vmess、VLESS
WindowsClash CFW (推荐) 已G自行搜索安装包SS、SSR、Trojan、Vmess、VLESS
macOSClashX 已G自行搜索安装包SS、SSR、Trojan、V2ray
macOSV2rayUSS、SSR、Trojan、V2ray
AndroidV2rayNGSS、Trojan、V2ray(Vmess、VLESS)、Xray
AndroidClashForAndroid (推荐) 已G自行搜索安装包SS、SSR、Trojan、Vmess、VLESS
AndroidNekoBoxForAndroid (推荐)VMess / VLESS / SSR / Trojan / Trojan-Go/ NaiveProxy / HTTP(S) / SOCKS5/etc.
IOSShadowrocket 小火箭 IOS非国区购买SS、SSR、Trojan、V2ray、VLESS
IOSQuantumult IOS非国区购买SS、SSR、Trojan、V2ray
IOSQuantumultX IOS非国区购买SS、SSR、Trojan、V2ray

Stargazers over time

Stargazers over time

声明

本项目仅限个人自己使用,禁止使用本项目进行营利和做其他违法事情,产生的一切后果本项目概不负责

回到顶部