org.jsoup.Connection.ignoreHttpErrors()方法的使用及代码示例

x33g5p2x  于2022-01-18 转载在 其他  
字(13.7k)|赞(0)|评价(0)|浏览(196)

本文整理了Java中org.jsoup.Connection.ignoreHttpErrors()方法的一些代码示例，展示了Connection.ignoreHttpErrors()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Connection.ignoreHttpErrors()方法的具体详情如下：
包路径：org.jsoup.Connection
类名称：Connection
方法名：ignoreHttpErrors

Connection.ignoreHttpErrors介绍

[英]Configures the connection to not throw exceptions when a HTTP error occurs. (4xx - 5xx, e.g. 404 or 500). By default this is false; an IOException is thrown if an error is encountered. If set to true, the response is populated with the error body, and the status message will reflect the error.
[中]将连接配置为在发生HTTP错误时不引发异常。（4xx-5xx，例如404或500）。默认情况下，这是错误的；如果遇到错误，将引发IOException。如果设置为true，则响应将填充错误正文，并且状态消息将反映错误。

代码示例

代码示例来源：origin: ahmetaa/zemberek-nlp

.ignoreHttpErrors(true)
.userAgent(
  "Mozilla/5.0 (Windows; U; WindowsNT 5.1; en-US; rv1.8.1.6) Gecko/20070725 Firefox/2.0.0.6")

代码示例来源：origin: pl.edu.icm.polindex/polindex-core

public Document postQuery(String url, Map<String,String> parameters, int timeoutMillis) throws IOException {
    Preconditions.checkArgument(StringUtils.isNotBlank(url));
    
    Date start = new Date();
    Connection con = Jsoup.connect(url)
               .timeout(timeoutMillis)
               .ignoreHttpErrors(false);
    
    if (parameters != null && !parameters.isEmpty()) {
      con.data(parameters);
    }
        
    logger.info(".. sending POST request to "+url+ " ...");
    Document doc = con.post();        
    
    Date stop = new Date();		
    
    logger.info(".. document from ["+url+"] received & parsed in "+ BeanStats.formatMillisPretty(stop.getTime() - start.getTime(), 0));
    
    return doc;
  }
}

代码示例来源：origin: BeelGroup/Docear-Desktop

private Map<String, String> requestNewCookie(String fileName) {		
  Map<String, String> cookies = null;
  try{
    Response response = getConnection(BaseURL).ignoreHttpErrors(true).execute();					
    cookies = response.cookies();
    String gsp = cookies.get("GSP");
    cookies.put("GSP", gsp + ":CF=4"); // :CF=4 enables the export to BibTex Link in the result list
    saveCookies(cookies, fileName);
  }catch(IOException e){
    logger.info(e.getMessage(), e);
  }
  return cookies;
}

代码示例来源：origin: YuanKJ-/JsCrawler

@Override
protected void processUrl(RequestModel model) {
  String url = model.getUrl();
  connection = Jsoup.connect(url).ignoreHttpErrors(true);
}

代码示例来源：origin: skydoves/SyncMarket

private static Elements getElementsFromNetwork(final String selects) {
  try {
    Elements elements = Jsoup.connect("https://play.google.com/store/apps/details?id=" + packageName)
        .timeout(timeout)
        .ignoreHttpErrors(true)
        .referrer("http://www.google.com").get()
        .select(selects);
    if(elements.size() != 0)
      return elements;
  } catch (IOException e) {
    e.printStackTrace();
  }
  return null;
}

代码示例来源：origin: Delsart/Bookster

.timeout(10000)
.ignoreContentType(true)
.ignoreHttpErrors(true)
.userAgent(Url.MOBBILE_AGENT)
.get()

代码示例来源：origin: Delsart/Bookster

@Override
  public void run() {
    try {
    Document document = Jsoup.connect(url)
          .timeout(10000)
          .ignoreContentType(true)
          .ignoreHttpErrors(true)
          .userAgent(Url.MOBBILE_AGENT)
          .get();
      Elements elements = document.select("#okBookShow > div.ok-book-base-info > div.row > div.col-sm-8.ok-book-info > div.ok-book-opt > div > div.col-md-5.ok-book-download > div a");
      if (elements != null) {
        for (Element element : elements) {
          urls.add(new DownloadBean(element.text(), element.attr("abs:href")));
        }
      }
    } catch (IOException e) {
      e.printStackTrace();
    }
    latch.countDown();
  }
});

代码示例来源：origin: Delsart/Bookster

@Override
  public void run() {
    String url = element.attr("abs:href");
    try {
      Document document = Jsoup.connect(url)
          .ignoreContentType(true)
          .ignoreHttpErrors(true)
          .userAgent(Url.MOBBILE_AGENT)
          .get();
      String name = document.select("#okBookShow > div.ok-book-base-info > div.row > div.col-sm-8.ok-book-info > div.ok-book-meta > h1").text();
      String time = "";
      String info = "";
      Elements elements = document.select("#okBookShow > div.ok-book-base-info > div.row > div.col-sm-8.ok-book-info > div.ok-book-meta > div.ok-book-desc > div.ok-book-meta-content").select("p");
      for (int i = 0; i < elements.size(); i++) {
        if (!elements.get(i).text().equals(""))
          info = info + elements.get(i).text() + "\n\n";
      }
      String category = document.select("#okBookShow > div.ok-book-base-info > div.row > div.col-sm-8.ok-book-info > div.ok-book-meta > div.ok-book-subjects").text();
      String status = "";
      String author = document.select("#okBookShow > div.ok-book-base-info > div.row > div.col-sm-8.ok-book-info > div.ok-book-meta > div.row > div > div").text();
      String words = "";
      String pic = document.select("#okBookShow > div.ok-book-base-info > div.row > div.col-sm-4 > div > img").attr("abs:src");
      NovelBean no = new NovelBean(name, time, info, category, status, author, words, pic, url);
      list.add(no);
    } catch (IOException e) {
      e.printStackTrace();
    }
    latch.countDown();
  }
});

代码示例来源：origin: Delsart/Bookster

@Override
  public void run() {
    Document document = null;
    try {
      document = Jsoup.connect(url)
          .timeout(10000)
          .ignoreContentType(true)
          .ignoreHttpErrors(true)
          .userAgent(Url.MOBBILE_AGENT)
          .get();
      Elements elements=document.select("body > div.wrap > div.content > div:nth-child(4) > div.panel-body a");
      for (Element element : elements) {
        urls.add(new DownloadBean(element.text(), element.attr("abs:href")));
      }
    } catch (IOException e) {
      e.printStackTrace();
    }
    latch.countDown();
  }
});

代码示例来源：origin: skydoves/SyncMarket

private static String getAttrFromNetwork(final String selects) {
  String result = "none";
  try {
    Elements elements = Jsoup.connect("https://play.google.com/store/apps/details?id=" + packageName)
        .timeout(timeout)
        .ignoreHttpErrors(true)
        .referrer("http://www.google.com").get()
        .select(selects);
    if(elements.size() != 0)
      result = elements.first().ownText();
  } catch (IOException e) {
    e.printStackTrace();
  }
  return result;
}

代码示例来源：origin: Delsart/Bookster

@Override
  public void run() {
    Document document = null;
    try {
      document = Jsoup.connect(url)
          .timeout(10000)
          .ignoreContentType(true)
          .ignoreHttpErrors(true)
          .userAgent(Url.MOBBILE_AGENT)
          .get();
      String u1 = "";
      String u1n = document.select("body > div:nth-child(4) > p:nth-child(7)").text();
      Elements elements = document.select("body > div.list").select("a");
      urls.add(new DownloadBean(u1n, u1));
      for (Element element : elements) {
        urls.add(new DownloadBean(element.text(), element.attr("abs:href")));
      }
    } catch (IOException e) {
      e.printStackTrace();
    }
    latch.countDown();
  }
});

代码示例来源：origin: Delsart/Bookster

@Override
  public void run() {
    String url = element.attr("abs:href");
    try {
      Document document = Jsoup.connect(url)
          .ignoreContentType(true)
          .ignoreHttpErrors(true)
          .userAgent(Url.MOBBILE_AGENT)
          .get();
      String name = document.select("body > div:nth-child(2) > h1").text();
      String time = document.select("body > div:nth-child(3) > li:nth-child(6)").text();
      String info = document.select("body > div.intro > li").text();
      String category = document.select("body > div:nth-child(3) > li:nth-child(2)").text();
      String status = document.select("body > div:nth-child(3) > li:nth-child(5)").text();
      String author = document.select("body > div:nth-child(3) > li:nth-child(1)").text();
      String words = document.select("body > div:nth-child(3) > li:nth-child(3)").text();
      String pic = "";
      NovelBean no = new NovelBean(name, time, info, category, status, author, words, pic, url);
      list.add(no);
    } catch (IOException e) {
      e.printStackTrace();
    }
    latch.countDown();
  }
});

代码示例来源：origin: delthas/JavaSkype

private String getToken(String token, String scope) throws IOException {
 Response response = Jsoup.connect(SERVER_HOSTNAME + "/oauth20_token.srf").data("client_id", "00000000480BC46C", "scope", scope, "grant_type", "refresh_token", "refresh_token", token).maxBodySize(100 * 1024 * 1024).timeout(10000).method(Method.POST).ignoreContentType(true).ignoreHttpErrors(true).execute();
 if (response.statusCode() != 200) {
  try {
   JSONObject json = new JSONObject(response.body());
   String errorDescription = json.getString("error_description");
   IOException ex = new IOException("Error while connecting to Live: token request error: " + errorDescription);
   logger.log(Level.SEVERE, "", ex);
   throw ex;
  } catch (JSONException | IllegalFormatException e) {
   IOException ex = new IOException("Error while connecting to Live: unknown token request error: " + response.body());
   logger.log(Level.SEVERE, "", ex);
   throw ex;
  }
 }
 try {
  JSONObject json = new JSONObject(response.body());
  String accessToken = json.getString("access_token");
  return accessToken;
 } catch (JSONException | IllegalFormatException e) {
  IOException ex = new IOException("Error while connecting to Live: failed reading token response: " + response.body());
  logger.log(Level.SEVERE, "", ex);
  throw ex;
 }
}

代码示例来源：origin: Delsart/Bookster

@Override
  public void run() {
    try {
      Document document = Jsoup.connect(url)
          .timeout(10000)
          .ignoreContentType(true)
          .ignoreHttpErrors(true)
          .userAgent(Url.MOBBILE_AGENT)
          .get();
    String u1 = document.select("body > div:nth-child(4) > div.show > div:nth-child(4) > div.showDown > ul > li:nth-child(1) > a").attr("abs:href");
    String u1n = document.select("body > div:nth-child(4) > div.show > div:nth-child(4) > div.showDown > ul > li:nth-child(1) > a").text();
    String u2 = document.select("body > div:nth-child(4) > div.show > div:nth-child(4) > div.showDown > ul > li:nth-child(2) > a").attr("abs:href");
    String u2n = document.select("body > div:nth-child(4) > div.show > div:nth-child(4) > div.showDown > ul > li:nth-child(2) > a").text();
    String u3 = document.select("body > div:nth-child(4) > div.show > div:nth-child(4) > div.showDown > ul > li:nth-child(3) > a").attr("abs:href");
    String u3n = document.select("body > div:nth-child(4) > div.show > div:nth-child(4) > div.showDown > ul > li:nth-child(3) > a").text();
    urls.add(new DownloadBean(u1n, u1));
    urls.add(new DownloadBean(u2n, u2));
    urls.add(new DownloadBean(u3n, u3));
    } catch (IOException e) {
      e.printStackTrace();
    }
    latch.countDown();
  }
});

代码示例来源：origin: delthas/JavaSkype

private Response sendRequest(Method method, String apiPath, boolean absoluteApiPath, String... keyval) throws IOException {
 String url = absoluteApiPath ? apiPath : SERVER_HOSTNAME + apiPath;
 Connection conn = Jsoup.connect(url).maxBodySize(100 * 1024 * 1024).timeout(10000).method(method).ignoreContentType(true).ignoreHttpErrors(true);
 logger.finest("Sending " + method + " request at " + url);
 if (skypeToken != null) {
  conn.header("X-Skypetoken", skypeToken);
 } else {
  logger.fine("No token sent for the request at: " + url);
 }
 conn.data(keyval);
 return conn.execute();
}

代码示例来源：origin: yy1193889747/springboot-demo

/**
 * 代理Ip爬取
 */
// @Scheduled(fixedRate = 10 * 60 * 1000)
public void ipProxy() throws IOException {
  Document doc = Jsoup.connect(URL_IP).userAgent(USER_AGENT).get();
  Elements ips = doc.select("body > div:nth-child(8) > ul > li:nth-child(2) > ul.l2").next();
  log.info("ip个数：{}", ips.size());
  for (int i = 0; i <= ips.size(); i++) {
    String ipaddr = ips.select("ul:nth-child(" + (i + 2) + ") > span:nth-child(1) > li").text();
    String proxy = ips.select("ul:nth-child(" + (i + 2) + ") > span:nth-child(2) > li").text();
    String speed = ips.select("ul:nth-child(" + (i + 2) + ") > span:nth-child(8) > li").text();
    log.info("ip: {}----端口: {} ----速度：{} ", ipaddr, proxy, speed);
    if (!"".equals(proxy)) {
      try {
        Jsoup.connect(URL_BLOG).proxy(ipaddr, Integer.parseInt(proxy)).ignoreHttpErrors(false).timeout(3000).get();
      } catch (IOException e) {
        log.info("不能用");
      }
    }
  }
}

代码示例来源：origin: zc-zh-001/ShadowSocks-Share

protected Connection getConnection(String url) {
  @SuppressWarnings("deprecation")
  Connection connection = Jsoup.connect(url)
      .userAgent("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.108 Safari/537.36")
      // .referrer("https://www.google.com/")
      .ignoreContentType(true)
      .followRedirects(true)
      .ignoreHttpErrors(true)
      .validateTLSCertificates(false)
      .timeout(TIME_OUT);
  if (isProxyEnable())
    connection.proxy(new Proxy(Proxy.Type.HTTP, new InetSocketAddress(getProxyHost(), getProxyPort())));
  return connection;
}

代码示例来源：origin: Delsart/Bookster

@Override
  public void run() {
    try {
      Elements elements = Jsoup.connect(url)
          .timeout(10000)
          .ignoreContentType(true)
          .ignoreHttpErrors(true)
          .userAgent(Url.MOBBILE_AGENT)
          .get()
          .select("body > div:nth-child(5)");
      String u1 = elements.select("li:nth-child(2) > a").attr("abs:href");
      String u1n = elements.select("li:nth-child(2) > a").text();
      String u2 = elements.select("li:nth-child(3) > a").attr("abs:href");
      String u2n = elements.select("li:nth-child(3) > a").text();
      urls.add(new DownloadBean(u1n, u1));
      urls.add(new DownloadBean(u2n, u2));
    } catch (IOException e) {
      e.printStackTrace();
    }
    latch.countDown();
  }
});

代码示例来源：origin: Delsart/Bookster

@Override
  public void run() {
    try {
      Elements elements = Jsoup.connect(url)
          .timeout(10000)
          .ignoreContentType(true)
          .ignoreHttpErrors(true)
          .userAgent(Url.MOBBILE_AGENT)
          .get()
          .select("body > div:nth-child(5)");
      String u1 = elements.select("li:nth-child(2) > a").attr("abs:href");
      String u1n = elements.select("li:nth-child(2) > a").text();
      String u2 = elements.select("li:nth-child(3) > a").attr("abs:href");
      String u2n = elements.select("li:nth-child(3) > a").text();
      urls.add(new DownloadBean(u1n, u1));
      urls.add(new DownloadBean(u2n, u2));
    } catch (IOException e) {
      e.printStackTrace();
    }
    latch.countDown();
  }
});

代码示例来源：origin: leftcoding/GankLy

@Override
  public void subscribe(ObservableEmitter<String> subscriber) throws Exception {
    try {
      Document doc = Jsoup.connect(url)
          .userAgent(USERAGENT)
          .timeout(timeout)
          .ignoreContentType(true)
          .ignoreHttpErrors(true)
          .get();
      String _url = null;
      if (doc != null) {
        doc = removeDivs(doc);
        _url = doc.html();
      }
      subscriber.onNext(_url);
      subscriber.onComplete();
    } catch (IOException e) {
      KLog.e(e);
      subscriber.onError(e);
    }
  }
})

内容来源于网络，如有侵权，请联系作者删除！

Connection

相关文章

    查看更多
    微信公众号

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com