org.jsoup.Connection.maxBodySize()方法的使用及代码示例

x33g5p2x  于2022-01-18 转载在 其他  
字(5.9k)|赞(0)|评价(0)|浏览(511)

本文整理了Java中org.jsoup.Connection.maxBodySize()方法的一些代码示例,展示了Connection.maxBodySize()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Connection.maxBodySize()方法的具体详情如下:
包路径:org.jsoup.Connection
类名称:Connection
方法名:maxBodySize

Connection.maxBodySize介绍

[英]Set the maximum bytes to read from the (uncompressed) connection into the body, before the connection is closed, and the input truncated. The default maximum is 1MB. A max size of zero is treated as an infinite amount (bounded only by your patience and the memory available on your machine).
[中]在连接关闭和输入被截断之前,设置从(未压缩的)连接读入正文的最大字节数。默认最大值为1MB。最大大小为零被视为无限量(仅受您的耐心和机器上可用内存的限制)。

代码示例

代码示例来源:origin: RipMeApp/ripme

.ignoreContentType(true)
.timeout(Utils.getConfigInteger("download.timeout", 60 * 1000))
.maxBodySize(1024 * 1024 * 100)
.execute();

代码示例来源:origin: RipMeApp/ripme

private static Document getDocument(String strUrl) throws IOException {
  return Jsoup.connect(strUrl)
              .userAgent(USER_AGENT)
              .timeout(10 * 1000)
              .maxBodySize(0)
              .get();
}

代码示例来源:origin: RipMeApp/ripme

private void defaultSettings() {
  this.retries = Utils.getConfigInteger("download.retries", 1);
  connection = Jsoup.connect(this.url);
  connection.userAgent(AbstractRipper.USER_AGENT);
  connection.method(Method.GET);
  connection.timeout(TIMEOUT);
  connection.maxBodySize(0);
}

代码示例来源:origin: small-dream/China_Province_City

public static void main(String[] args) {
  try {
    //2018年11月中华人民共和国县以上行政区划代码网页
    Document doc = Jsoup.connect("http://www.mca.gov.cn/article/sj/xzqh/2018/201804-12/20181101021046.html").maxBodySize(0).get();
    Elements elements = doc.getElementsByClass("xl7024197");
    List<String> stringList = elements.eachText();
    List<String> stringName = new ArrayList<String>();
    List<String> stringCode = new ArrayList<String>();
    for (int i = 0; i < stringList.size(); i++) {
      if (i % 2 == 0) {
        //地区代码
        stringCode.add(stringList.get(i));
      } else {
        //地区名字
        stringName.add(stringList.get(i));
      }
    }
    //正常情况 两个 list size 应该 一样
    System.out.println("stringName  size= " + stringName.size() + "   stringCode   size= " + stringCode.size());
    if (stringName.size() != stringCode.size()) {
      throw new RuntimeException("数据错误");
    }
    List<Province> provinceList = processData(stringName, stringCode);
    String path = FileUtils.getProjectDir() + "/2018年11月中华人民共和国县以上行政区划代码" + ".json";
    JSONFormatUtils.jsonWriter(provinceList, path);
  } catch (IOException e) {
    e.printStackTrace();
  }
}

代码示例来源:origin: ysc/HtmlExtractor

Connection.Response response = Jsoup.connect(url).maxBodySize(0).ignoreContentType(true).timeout(timeout).execute();

代码示例来源:origin: ysc/HtmlExtractor

Connection.Response response = Jsoup.connect(href).maxBodySize(0).ignoreContentType(true).timeout(timeout).execute();

代码示例来源:origin: ysc/HtmlExtractor

Connection.Response response = Jsoup.connect(resource).maxBodySize(0).ignoreContentType(true).timeout(timeout).execute();

代码示例来源:origin: dimtion/Shaarlier

try {
  pageResp = Jsoup.connect(url)
      .maxBodySize(LOAD_TITLE_MAX_BODY_SIZE) // Hopefully we won't need more data
      .followRedirects(true)
      .execute()

代码示例来源:origin: delthas/JavaSkype

private String getToken(String token, String scope) throws IOException {
 Response response = Jsoup.connect(SERVER_HOSTNAME + "/oauth20_token.srf").data("client_id", "00000000480BC46C", "scope", scope, "grant_type", "refresh_token", "refresh_token", token).maxBodySize(100 * 1024 * 1024).timeout(10000).method(Method.POST).ignoreContentType(true).ignoreHttpErrors(true).execute();
 if (response.statusCode() != 200) {
  try {
   JSONObject json = new JSONObject(response.body());
   String errorDescription = json.getString("error_description");
   IOException ex = new IOException("Error while connecting to Live: token request error: " + errorDescription);
   logger.log(Level.SEVERE, "", ex);
   throw ex;
  } catch (JSONException | IllegalFormatException e) {
   IOException ex = new IOException("Error while connecting to Live: unknown token request error: " + response.body());
   logger.log(Level.SEVERE, "", ex);
   throw ex;
  }
 }
 try {
  JSONObject json = new JSONObject(response.body());
  String accessToken = json.getString("access_token");
  return accessToken;
 } catch (JSONException | IllegalFormatException e) {
  IOException ex = new IOException("Error while connecting to Live: failed reading token response: " + response.body());
  logger.log(Level.SEVERE, "", ex);
  throw ex;
 }
}

代码示例来源:origin: delthas/JavaSkype

private Response sendRequest(Method method, String apiPath, boolean absoluteApiPath, String... keyval) throws IOException {
 String url = absoluteApiPath ? apiPath : SERVER_HOSTNAME + apiPath;
 Connection conn = Jsoup.connect(url).maxBodySize(100 * 1024 * 1024).timeout(10000).method(method).ignoreContentType(true).ignoreHttpErrors(true);
 logger.finest("Sending " + method + " request at " + url);
 if (skypeToken != null) {
  conn.header("X-Skypetoken", skypeToken);
 } else {
  logger.fine("No token sent for the request at: " + url);
 }
 conn.data(keyval);
 return conn.execute();
}

代码示例来源:origin: delthas/JavaSkype

public synchronized long refreshTokens() throws IOException {
 logger.finer("Refreshing tokens");
 Response authorize = Jsoup.connect(SERVER_HOSTNAME + "/oauth20_authorize.srf?client_id=00000000480BC46C&scope=service%3A%3Askype.com%3A%3AMBI_SSL&response_type=token&redirect_uri=https%3A%2F%2Flogin.live.com%2Foauth20_desktop.srf&state=999&locale=en").maxBodySize(100 * 1024 * 1024).timeout(10000).method(Method.GET).ignoreContentType(true).ignoreHttpErrors(true).execute();
 Response post = Jsoup.connect(postUrl).data("PPFT", PPFT, "login", username, "passwd", password).cookie("MSPOK", MSPOK).maxBodySize(100 * 1024 * 1024).timeout(10000).method(Method.POST).followRedirects(false).ignoreContentType(true).ignoreHttpErrors(true).execute();
 if (post.statusCode() != 302) {
  int index = post.body().indexOf("sErrTxt:'");

代码示例来源:origin: io.github.christian-draeger/page-content-tester

.maxBodySize(0)
.timeout(params.getTimeout());

代码示例来源:origin: xuxueli/xxl-crawler

conn.maxBodySize(0);    // 取消默认1M限制

代码示例来源:origin: xuxueli/xxl-crawler

conn.maxBodySize(0);    // 取消默认1M限制

相关文章