csv Python块写入excel

pkln4tw6 于 2022-12-06 发布在 Python

关注(0)|答案(2)|浏览(135)

我是Python的新手，我在实践中学习。
此时，我的代码运行得相当慢，而且每次运行它所花费的时间似乎越来越长。
我们的想法是以CSV格式下载员工列表，然后通过在特定页面上运行员工ID并将其写入Excel文件来检查每个员工ID的位置。
我们每天有大约600名员工在现场，我需要找到他们的位置，并保持每2-4分钟刷新一次。
编辑：
为了让大家更好地理解，我准备了一个CSV文件（TOT.CSV），其中包含员工ID、姓名和我在现场的同事的其他信息。
为了得到他们的位置，我需要从该CSV文件运行每个员工ID通过https://guided-coaching-dub.corp.amazon.com/api/employee-location-svc/GetLastSeenLocationOfEmployee?employeeId= 1对1，同时将其写入另一个CSV文件（Location.csv）。现在，它在大约10分钟内完成，我想知道我做的方式是否是最好的可能的方式，或者是否有其他东西，我可以尝试。
我的代码如下所示：

# GET EMPLOYEE ID FROM THE CSV

data = read_csv("Z:\\_Tracker\\Dump\\attendance\\TOT.csv")

# converting column data to list
TOT_employeeID = data['Employee ID'].tolist()

# Clean the Location Sheet

with open("Z:\\_Tracker\\Dump\\attendance\\Location.csv", "w") as f:
    pass

print("Previous Location data cleared ... ")

# go through EACH employee ID to find out location

for x in TOT_employeeID:
    driver.get(
        "https://guided-coaching-dub.corp.amazon.com/api/employee-location-svc/GetLastSeenLocationOfEmployee?employeeId=" + x)
    print("Getting Location data for EmployeeID: " + x)
    locData = driver.find_element(By.TAG_NAME, 'body').text
    aaData = str(locData)
    realLoc = aaData.split('"')

    # write to excel
    with open("Z:\\_Tracker\\Dump\\attendance\\Location.csv",
              "a") as f:
        writer = csv.writer(f)
        writer.writerow(realLoc)

time.sleep(5)
print("Employee Location data downloaded...")

有什么办法能让我更快一点吗？
提前感谢您！
祝你好运Alex

csv

来源：https://stackoverflow.com/questions/74035632/python-chunks-write-to-excel

2条答案

按热度按时间

8fq7wneg1#

就像这样

import concurrent.futures

def process_data(data: pd.DataFrame) -> None:
    associates = data['Employee ID'].unique()
    with concurrent.futures.ProcessPoolExecutor() as executer:
        executer.map(get_location, associates)

def get_location(associate: str) -> None:
    driver.get(
        "https://guided-coaching-dub.corp.amazon.com/api/employee-location-svc/GetLastSeenLocationOfEmployee?"
        f"employeeId={associate}")
    print(f"Getting Location data for EmployeeID: {associate}")
    realLoc = str(driver.find_element(By.TAG_NAME, 'body').text).split('"')

    with open("Z:\\_Tracker\\Dump\\attendance\\Location.csv", "a") as f:
        writer = csv.writer(f)
        writer.writerow(realLoc)

if __name__ == "__main__":
    data = read_csv("Z:\\_Tracker\\Dump\\attendance\\TOT.csv")
    process_data(data)

赞(0）回复(0）举报 2022-12-06

1l5u6lss2#

您可以尝试将阅读信息和将信息写入CSV文件的步骤分开，如下所示：

# Get Employee Location Information
# Create list for employee information, to be used below
employee_Locations = []
 
for x in TOT_employeeID:
    driver.get("https://guided-coaching-dub.corp.amazon.com/api/employee-location-svc/GetLastSeenLocationOfEmployee?employeeId=" + x)
    print("Getting Location data for EmployeeID: " + x)
    locData = driver.find_element(By.TAG_NAME, 'body').text
    aaData = str(locData)
    realLoc = [aaData.split('"')]
    employee_Locations.extend(realLoc)
            
# Write to excel - Try this as a separate step
with open("Z:\\_Tracker\\Dump\\attendance\\Location.csv","a") as f:
    writer = csv.writer(f, delimiter='\n')
    writer.writerow(employee_Locations)
            
print("Employee Location data downloaded...")

通过先收集所有信息，然后写入CSV文件，您可能会看到一些性能提升

赞(0）回复(0）举报 2022-12-06

我来回答

csv Python块写入excel

2条答案

相关问题

热门标签

最新问答